[2024-07-15 06:41:47.064535] Starting SPDK v24.05.1-pre git sha1 5fa2f5086 / DPDK 23.11.0 initialization... [2024-07-15 06:41:47.064635] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2424547 ] Set seed to 1721018507062201 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 06:41:47.162937] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 06:41:47.205152] app.c: 604:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 06:41:47.205205] app.c: 605:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2424547' to capture a snapshot of events at runtime. [2024-07-15 06:41:47.205215] app.c: 610:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 06:41:47.205226] app.c: 611:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 06:41:47.205233] app.c: 612:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2424547 for offline analysis/debug. [2024-07-15 06:41:47.205325] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 06:41:51 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/07/15-06:41:54 ... thread 0: (1000000,1000000) ops and (358212.5,358212.5) ops/second in (2.791639,2.791639) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 9/6 567.03 MB 18.9 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 350.7 2.34 2.13 12 0.195 0 0 0.0 0.0 L2 3/2 251.75 MB 0.0 0.4 0.2 0.2 0.4 0.2 0.2 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 9/0 741.47 MB 0.7 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 116/0 9.60 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1044/0 85.82 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1181/8 96.94 GB 0.0 0.4 0.2 0.2 1.2 1.0 0.3 1.5 98.6 312.3 3.83 3.57 14 0.274 383K 197 0.0 0.0 Int 0/0 0.00 KB 0.0 0.4 0.2 0.2 1.2 1.0 0.2 1.5 98.6 312.3 3.83 3.57 14 0.274 383K 197 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 0.0 0.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 350.7 2.34 2.13 12 0.195 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 3.0 total, 2.8 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.17 GB write, 400.71 MB/s write, 0.37 GB read, 126.55 MB/s read, 3.8 seconds Interval compaction: 1.17 GB write, 428.46 MB/s write, 0.37 GB read, 135.31 MB/s read, 3.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 3, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 3, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 101285 Average: 1.8980 StdDev: 20.74 Min: 1 Median: 1.2016 Max: 3515 Percentiles: P50: 1.20 P75: 1.65 P99: 3.90 P99.9: 9.79 P99.99: 278.32 ------------------------------------------------------ [ 0, 1 ] 39281 38.783% 38.783% ######## ( 1, 2 ] 56349 55.634% 94.417% ########### ( 2, 3 ] 3598 3.552% 97.969% # ( 3, 4 ] 1156 1.141% 99.110% ( 4, 6 ] 601 0.593% 99.704% ( 6, 10 ] 212 0.209% 99.913% ( 10, 15 ] 54 0.053% 99.966% ( 15, 22 ] 7 0.007% 99.973% ( 22, 34 ] 2 0.002% 99.975% ( 34, 51 ] 2 0.002% 99.977% ( 51, 76 ] 1 0.001% 99.978% ( 110, 170 ] 8 0.008% 99.986% ( 170, 250 ] 5 0.005% 99.991% ( 250, 380 ] 4 0.004% 99.995% ( 380, 580 ] 1 0.001% 99.996% ( 580, 870 ] 1 0.001% 99.997% ( 1900, 2900 ] 2 0.002% 99.999% ( 2900, 4400 ] 3 0.003% 100.002% ** Level 1 read latency histogram (micros): Count: 5 Average: 1076.4000 StdDev: 1071.95 Min: 68 Median: 315.0000 Max: 2421 Percentiles: P50: 315.00 P75: 2275.00 P99: 2421.00 P99.9: 2421.00 P99.99: 2421.00 ------------------------------------------------------ ( 51, 76 ] 1 20.000% 20.000% #### ( 170, 250 ] 1 20.000% 40.000% #### ( 250, 380 ] 1 20.000% 60.000% #### ( 1900, 2900 ] 2 40.000% 100.000% ######## ** Level 2 read latency histogram (micros): Count: 50640 Average: 1.8684 StdDev: 8.06 Min: 1 Median: 1.2503 Max: 1118 Percentiles: P50: 1.25 P75: 1.69 P99: 5.09 P99.9: 11.41 P99.99: 442.40 ------------------------------------------------------ [ 0, 1 ] 18105 35.752% 35.752% ####### ( 1, 2 ] 28822 56.915% 92.668% ########### ( 2, 3 ] 2020 3.989% 96.657% # ( 3, 4 ] 900 1.777% 98.434% ( 4, 6 ] 529 1.045% 99.479% ( 6, 10 ] 200 0.395% 99.874% ( 10, 15 ] 51 0.101% 99.974% ( 15, 22 ] 3 0.006% 99.980% ( 22, 34 ] 3 0.006% 99.986% ( 51, 76 ] 2 0.004% 99.990% ( 380, 580 ] 3 0.006% 99.996% ( 580, 870 ] 1 0.002% 99.998% ( 870, 1300 ] 2 0.004% 100.002% ** Level 3 read latency histogram (micros): Count: 40 Average: 1363.8750 StdDev: 1096.19 Min: 200 Median: 870.0000 Max: 3622 Percentiles: P50: 870.00 P75: 2275.00 P99: 3622.00 P99.9: 3622.00 P99.99: 3622.00 ------------------------------------------------------ ( 170, 250 ] 3 7.500% 7.500% ## ( 250, 380 ] 7 17.500% 25.000% #### ( 380, 580 ] 7 17.500% 42.500% #### ( 580, 870 ] 3 7.500% 50.000% ## ( 870, 1300 ] 3 7.500% 57.500% ## ( 1300, 1900 ] 4 10.000% 67.500% ## ( 1900, 2900 ] 8 20.000% 87.500% #### ( 2900, 4400 ] 5 12.500% 100.000% ### ** Level 4 read latency histogram (micros): Count: 35 Average: 1664.2571 StdDev: 1044.76 Min: 510 Median: 1390.0000 Max: 4479 Percentiles: P50: 1390.00 P75: 1950.00 P99: 4479.00 P99.9: 4479.00 P99.99: 4479.00 ------------------------------------------------------ ( 380, 580 ] 3 8.571% 8.571% ## ( 580, 870 ] 5 14.286% 22.857% ### ( 870, 1300 ] 8 22.857% 45.714% ##### ( 1300, 1900 ] 10 28.571% 74.286% ###### ( 1900, 2900 ] 5 14.286% 88.571% ### ( 2900, 4400 ] 3 8.571% 97.143% ## ( 4400, 6600 ] 1 2.857% 100.000% # ** DB Stats ** Uptime(secs): 3.0 total, 2.8 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 329.52 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.241 H:M:S, 8.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 352.24 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.241 H:M:S, 8.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-06:41:57 ... thread 0: (1000000,2000000) ops and (329411.6,343208.9) ops/second in (3.035716,5.827355) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 11/8 1.17 GB 62.9 0.6 0.0 0.6 2.3 1.8 0.0 1.3 77.3 326.4 7.33 6.77 28 0.262 575K 593 0.0 0.0 L1 1/0 2.92 MB 0.3 0.4 0.4 0.0 0.4 0.4 0.2 1.0 238.4 238.0 1.59 1.54 1 1.586 383K 591 0.0 0.0 L2 2/0 55.35 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.5 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 42/0 998.90 MB 1.0 0.8 0.6 0.2 0.8 0.6 0.1 1.4 204.6 204.4 3.89 3.45 3 1.296 808K 804 0.0 0.0 L4 144/0 9.53 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1248/8 97.92 GB 0.0 2.1 1.1 1.0 3.9 2.9 1.6 2.2 148.1 275.7 14.30 13.20 34 0.421 2151K 2185 0.0 0.0 Int 0/0 0.00 KB 0.0 2.1 1.1 1.0 3.9 2.9 1.5 2.2 148.1 275.7 14.30 13.20 34 0.421 2151K 2185 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.1 1.1 1.0 2.1 1.1 0.0 0.0 232.2 232.0 9.12 8.55 8 1.141 2151K 2185 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 352.7 5.18 4.65 26 0.199 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.0 total, 5.8 interval Flush(GB): cumulative 1.784, interval 1.784 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 3.85 GB write, 654.82 MB/s write, 2.07 GB read, 351.83 MB/s read, 14.3 seconds Interval compaction: 3.85 GB write, 676.56 MB/s write, 2.07 GB read, 363.51 MB/s read, 14.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 6, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 6, interval: 6 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 387197 Average: 2.1630 StdDev: 13.85 Min: 1 Median: 1.4560 Max: 3515 Percentiles: P50: 1.46 P75: 1.85 P99: 5.63 P99.9: 11.09 P99.99: 19.12 ------------------------------------------------------ [ 0, 1 ] 81687 21.097% 21.097% #### ( 1, 2 ] 245445 63.390% 84.487% ############# ( 2, 3 ] 35841 9.257% 93.744% ## ( 3, 4 ] 13468 3.478% 97.222% # ( 4, 6 ] 8458 2.184% 99.407% ( 6, 10 ] 1825 0.471% 99.878% ( 10, 15 ] 409 0.106% 99.983% ( 15, 22 ] 48 0.012% 99.996% ( 22, 34 ] 5 0.001% 99.997% ( 34, 51 ] 3 0.001% 99.998% ( 51, 76 ] 4 0.001% 99.999% ( 110, 170 ] 14 0.004% 100.003% ( 170, 250 ] 12 0.003% 100.006% ( 250, 380 ] 9 0.002% 100.008% ( 380, 580 ] 3 0.001% 100.009% ( 580, 870 ] 6 0.002% 100.010% ( 870, 1300 ] 3 0.001% 100.011% ( 1300, 1900 ] 2 0.001% 100.012% ( 1900, 2900 ] 2 0.001% 100.012% ( 2900, 4400 ] 5 0.001% 100.013% ** Level 1 read latency histogram (micros): Count: 81484 Average: 2.5272 StdDev: 13.12 Min: 1 Median: 1.6577 Max: 2421 Percentiles: P50: 1.66 P75: 2.17 P99: 5.99 P99.9: 13.09 P99.99: 18.00 ------------------------------------------------------ [ 0, 1 ] 6235 7.652% 7.652% ## ( 1, 2 ] 52468 64.391% 72.042% ############# ( 2, 3 ] 13912 17.073% 89.116% ### ( 3, 4 ] 4462 5.476% 94.592% # ( 4, 6 ] 3616 4.438% 99.029% # ( 6, 10 ] 602 0.739% 99.768% ( 10, 15 ] 174 0.214% 99.982% ( 15, 22 ] 16 0.020% 100.001% ( 22, 34 ] 2 0.002% 100.004% ( 34, 51 ] 1 0.001% 100.005% ( 51, 76 ] 1 0.001% 100.006% ( 110, 170 ] 1 0.001% 100.007% ( 170, 250 ] 2 0.002% 100.010% ( 250, 380 ] 3 0.004% 100.013% ( 380, 580 ] 2 0.002% 100.016% ( 580, 870 ] 1 0.001% 100.017% ( 870, 1300 ] 1 0.001% 100.018% ( 1900, 2900 ] 2 0.002% 100.021% ** Level 2 read latency histogram (micros): Count: 95433 Average: 2.0099 StdDev: 5.91 Min: 1 Median: 1.3912 Max: 1118 Percentiles: P50: 1.39 P75: 1.79 P99: 5.52 P99.9: 10.73 P99.99: 27.83 ------------------------------------------------------ [ 0, 1 ] 24545 25.720% 25.720% ##### ( 1, 2 ] 59231 62.066% 87.785% ############ ( 2, 3 ] 6520 6.832% 94.617% # ( 3, 4 ] 2843 2.979% 97.596% # ( 4, 6 ] 1760 1.844% 99.440% ( 6, 10 ] 425 0.445% 99.886% ( 10, 15 ] 93 0.097% 99.983% ( 15, 22 ] 5 0.005% 99.988% ( 22, 34 ] 3 0.003% 99.992% ( 51, 76 ] 2 0.002% 99.994% ( 380, 580 ] 3 0.003% 99.997% ( 580, 870 ] 1 0.001% 99.998% ( 870, 1300 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57468 Average: 3.4841 StdDev: 46.68 Min: 0 Median: 1.6585 Max: 3622 Percentiles: P50: 1.66 P75: 2.19 P99: 6.85 P99.9: 40.00 P99.99: 2056.65 ------------------------------------------------------ [ 0, 1 ] 4626 8.050% 8.050% ## ( 1, 2 ] 36611 63.707% 71.756% ############# ( 2, 3 ] 9948 17.311% 89.067% ### ( 3, 4 ] 3121 5.431% 94.498% # ( 4, 6 ] 2493 4.338% 98.836% # ( 6, 10 ] 442 0.769% 99.605% ( 10, 15 ] 117 0.204% 99.809% ( 15, 22 ] 20 0.035% 99.843% ( 22, 34 ] 29 0.050% 99.894% ( 34, 51 ] 10 0.017% 99.911% ( 110, 170 ] 2 0.003% 99.915% ( 170, 250 ] 8 0.014% 99.929% ( 250, 380 ] 12 0.021% 99.950% ( 380, 580 ] 9 0.016% 99.965% ( 580, 870 ] 5 0.009% 99.974% ( 870, 1300 ] 4 0.007% 99.981% ( 1300, 1900 ] 4 0.007% 99.988% ( 1900, 2900 ] 8 0.014% 100.002% ( 2900, 4400 ] 5 0.009% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1664.2571 StdDev: 1044.76 Min: 510 Median: 1390.0000 Max: 4479 Percentiles: P50: 1390.00 P75: 1950.00 P99: 4479.00 P99.9: 4479.00 P99.99: 4479.00 ------------------------------------------------------ ( 380, 580 ] 3 8.571% 8.571% ## ( 580, 870 ] 5 14.286% 22.857% ### ( 870, 1300 ] 8 22.857% 45.714% ##### ( 1300, 1900 ] 10 28.571% 74.286% ###### ( 1900, 2900 ] 5 14.286% 88.571% ### ( 2900, 4400 ] 3 8.571% 97.143% ## ( 4400, 6600 ] 1 2.857% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 738.1000 StdDev: 827.14 Min: 97 Median: 380.0000 Max: 2698 Percentiles: P50: 380.00 P75: 870.00 P99: 2698.00 P99.9: 2698.00 P99.99: 2698.00 ------------------------------------------------------ ( 76, 110 ] 4 20.000% 20.000% #### ( 110, 170 ] 2 10.000% 30.000% ## ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 2 10.000% 50.000% ## ( 380, 580 ] 3 15.000% 65.000% ### ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 2 10.000% 85.000% ## ( 1900, 2900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 6.0 total, 3.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 326.84 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.655 H:M:S, 10.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 324.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.415 H:M:S, 13.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-06:42:00 ... thread 0: (1000000,3000000) ops and (341493.0,342635.0) ops/second in (2.928318,8.755673) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/4 2.15 GB 188.8 1.7 0.0 1.7 4.4 2.8 0.0 1.6 119.0 317.3 14.29 13.38 46 0.311 1727K 1775 0.0 0.0 L1 1/0 2.92 MB 0.3 0.4 0.4 0.0 0.4 0.4 0.2 1.0 238.4 238.0 1.59 1.54 1 1.586 383K 591 0.0 0.0 L2 2/0 55.35 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.5 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 42/0 998.90 MB 1.0 0.8 0.6 0.2 0.8 0.6 0.1 1.4 204.6 204.4 3.89 3.45 3 1.296 808K 804 0.0 0.0 L4 144/0 9.53 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1250/4 98.90 GB 0.0 3.2 1.1 2.1 5.9 3.9 1.6 2.1 153.0 286.2 21.26 19.82 52 0.409 3302K 3367 0.0 0.0 Int 0/0 0.00 KB 0.0 3.2 1.1 2.1 5.9 3.9 1.5 2.1 153.0 286.2 21.26 19.82 52 0.409 3302K 3367 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 3.2 1.1 2.1 3.2 1.1 0.0 0.0 243.6 243.3 13.36 12.70 12 1.113 3302K 3367 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 358.6 7.90 7.11 40 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 9.0 total, 8.8 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.94 GB write, 679.75 MB/s write, 3.18 GB read, 363.42 MB/s read, 21.3 seconds Interval compaction: 5.94 GB write, 694.77 MB/s write, 3.18 GB read, 371.45 MB/s read, 21.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 10, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 10, interval: 10 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 633560 Average: 2.0570 StdDev: 12.61 Min: 1 Median: 1.3916 Max: 3515 Percentiles: P50: 1.39 P75: 1.80 P99: 5.39 P99.9: 10.77 P99.99: 21.43 ------------------------------------------------------ [ 0, 1 ] 163054 25.736% 25.736% ##### ( 1, 2 ] 392570 61.963% 87.699% ############ ( 2, 3 ] 47017 7.421% 95.120% # ( 3, 4 ] 17476 2.758% 97.878% # ( 4, 6 ] 10262 1.620% 99.498% ( 6, 10 ] 2458 0.388% 99.886% ( 10, 15 ] 589 0.093% 99.979% ( 15, 22 ] 78 0.012% 99.991% ( 22, 34 ] 16 0.003% 99.994% ( 34, 51 ] 7 0.001% 99.995% ( 51, 76 ] 7 0.001% 99.996% ( 110, 170 ] 22 0.003% 99.999% ( 170, 250 ] 15 0.002% 100.002% ( 250, 380 ] 16 0.003% 100.004% ( 380, 580 ] 6 0.001% 100.005% ( 580, 870 ] 12 0.002% 100.007% ( 870, 1300 ] 7 0.001% 100.008% ( 1300, 1900 ] 6 0.001% 100.009% ( 1900, 2900 ] 3 0.000% 100.010% ( 2900, 4400 ] 5 0.001% 100.010% ** Level 1 read latency histogram (micros): Count: 81484 Average: 2.5272 StdDev: 13.12 Min: 1 Median: 1.6577 Max: 2421 Percentiles: P50: 1.66 P75: 2.17 P99: 5.99 P99.9: 13.09 P99.99: 18.00 ------------------------------------------------------ [ 0, 1 ] 6235 7.652% 7.652% ## ( 1, 2 ] 52468 64.391% 72.042% ############# ( 2, 3 ] 13912 17.073% 89.116% ### ( 3, 4 ] 4462 5.476% 94.592% # ( 4, 6 ] 3616 4.438% 99.029% # ( 6, 10 ] 602 0.739% 99.768% ( 10, 15 ] 174 0.214% 99.982% ( 15, 22 ] 16 0.020% 100.001% ( 22, 34 ] 2 0.002% 100.004% ( 34, 51 ] 1 0.001% 100.005% ( 51, 76 ] 1 0.001% 100.006% ( 110, 170 ] 1 0.001% 100.007% ( 170, 250 ] 2 0.002% 100.010% ( 250, 380 ] 3 0.004% 100.013% ( 380, 580 ] 2 0.002% 100.016% ( 580, 870 ] 1 0.001% 100.017% ( 870, 1300 ] 1 0.001% 100.018% ( 1900, 2900 ] 2 0.002% 100.021% ** Level 2 read latency histogram (micros): Count: 95433 Average: 2.0099 StdDev: 5.91 Min: 1 Median: 1.3912 Max: 1118 Percentiles: P50: 1.39 P75: 1.79 P99: 5.52 P99.9: 10.73 P99.99: 27.83 ------------------------------------------------------ [ 0, 1 ] 24545 25.720% 25.720% ##### ( 1, 2 ] 59231 62.066% 87.785% ############ ( 2, 3 ] 6520 6.832% 94.617% # ( 3, 4 ] 2843 2.979% 97.596% # ( 4, 6 ] 1760 1.844% 99.440% ( 6, 10 ] 425 0.445% 99.886% ( 10, 15 ] 93 0.097% 99.983% ( 15, 22 ] 5 0.005% 99.988% ( 22, 34 ] 3 0.003% 99.992% ( 51, 76 ] 2 0.002% 99.994% ( 380, 580 ] 3 0.003% 99.997% ( 580, 870 ] 1 0.001% 99.998% ( 870, 1300 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57468 Average: 3.4841 StdDev: 46.68 Min: 0 Median: 1.6585 Max: 3622 Percentiles: P50: 1.66 P75: 2.19 P99: 6.85 P99.9: 40.00 P99.99: 2056.65 ------------------------------------------------------ [ 0, 1 ] 4626 8.050% 8.050% ## ( 1, 2 ] 36611 63.707% 71.756% ############# ( 2, 3 ] 9948 17.311% 89.067% ### ( 3, 4 ] 3121 5.431% 94.498% # ( 4, 6 ] 2493 4.338% 98.836% # ( 6, 10 ] 442 0.769% 99.605% ( 10, 15 ] 117 0.204% 99.809% ( 15, 22 ] 20 0.035% 99.843% ( 22, 34 ] 29 0.050% 99.894% ( 34, 51 ] 10 0.017% 99.911% ( 110, 170 ] 2 0.003% 99.915% ( 170, 250 ] 8 0.014% 99.929% ( 250, 380 ] 12 0.021% 99.950% ( 380, 580 ] 9 0.016% 99.965% ( 580, 870 ] 5 0.009% 99.974% ( 870, 1300 ] 4 0.007% 99.981% ( 1300, 1900 ] 4 0.007% 99.988% ( 1900, 2900 ] 8 0.014% 100.002% ( 2900, 4400 ] 5 0.009% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1664.2571 StdDev: 1044.76 Min: 510 Median: 1390.0000 Max: 4479 Percentiles: P50: 1390.00 P75: 1950.00 P99: 4479.00 P99.9: 4479.00 P99.99: 4479.00 ------------------------------------------------------ ( 380, 580 ] 3 8.571% 8.571% ## ( 580, 870 ] 5 14.286% 22.857% ### ( 870, 1300 ] 8 22.857% 45.714% ##### ( 1300, 1900 ] 10 28.571% 74.286% ###### ( 1900, 2900 ] 5 14.286% 88.571% ### ( 2900, 4400 ] 3 8.571% 97.143% ## ( 4400, 6600 ] 1 2.857% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 738.1000 StdDev: 827.14 Min: 97 Median: 380.0000 Max: 2698 Percentiles: P50: 380.00 P75: 870.00 P99: 2698.00 P99.9: 2698.00 P99.99: 2698.00 ------------------------------------------------------ ( 76, 110 ] 4 20.000% 20.000% #### ( 110, 170 ] 2 10.000% 30.000% ## ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 2 10.000% 50.000% ## ( 380, 580 ] 3 15.000% 65.000% ### ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 2 10.000% 85.000% ## ( 1900, 2900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 9.0 total, 2.9 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 329.87 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.044 H:M:S, 11.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 336.10 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.388 H:M:S, 13.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-06:42:03 ... thread 0: (1000000,4000000) ops and (342554.9,342615.0) ops/second in (2.919240,11.674913) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/14 3.07 GB 0.0 2.8 0.0 2.8 6.5 3.7 0.0 1.7 134.2 313.1 21.12 19.82 63 0.335 2878K 3001 0.0 0.0 L1 1/1 2.92 MB 0.0 0.4 0.4 0.0 0.4 0.4 0.2 1.0 238.4 238.0 1.59 1.54 1 1.586 383K 591 0.0 0.0 L2 2/0 55.35 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.5 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 42/0 998.90 MB 1.0 0.8 0.6 0.2 0.8 0.6 0.1 1.4 204.6 204.4 3.89 3.45 3 1.296 808K 804 0.0 0.0 L4 144/0 9.53 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1251/15 99.82 GB 0.0 4.3 1.1 3.2 8.0 4.8 1.6 2.2 156.2 290.6 28.09 26.25 69 0.407 4454K 4593 0.0 0.0 Int 0/0 0.00 KB 0.0 4.3 1.1 3.2 8.0 4.8 1.5 2.2 156.2 290.6 28.09 26.25 69 0.407 4454K 4593 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.3 1.1 3.2 4.3 1.1 0.0 0.0 248.9 248.6 17.62 16.86 16 1.102 4454K 4593 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 361.2 10.46 9.39 53 0.197 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.9 total, 11.7 interval Flush(GB): cumulative 3.691, interval 3.691 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.97 GB write, 687.61 MB/s write, 4.28 GB read, 369.57 MB/s read, 28.1 seconds Interval compaction: 7.97 GB write, 699.00 MB/s write, 4.28 GB read, 375.69 MB/s read, 28.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 13, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 13, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 883718 Average: 2.0501 StdDev: 12.98 Min: 1 Median: 1.3765 Max: 3946 Percentiles: P50: 1.38 P75: 1.78 P99: 5.28 P99.9: 11.25 P99.99: 121.03 ------------------------------------------------------ [ 0, 1 ] 236466 26.758% 26.758% ##### ( 1, 2 ] 545556 61.734% 88.492% ############ ( 2, 3 ] 62031 7.019% 95.512% # ( 3, 4 ] 22746 2.574% 98.085% # ( 4, 6 ] 12625 1.429% 99.514% ( 6, 10 ] 3215 0.364% 99.878% ( 10, 15 ] 779 0.088% 99.966% ( 15, 22 ] 101 0.011% 99.977% ( 22, 34 ] 19 0.002% 99.980% ( 34, 51 ] 9 0.001% 99.981% ( 51, 76 ] 12 0.001% 99.982% ( 76, 110 ] 55 0.006% 99.988% ( 110, 170 ] 85 0.010% 99.998% ( 170, 250 ] 25 0.003% 100.001% ( 250, 380 ] 33 0.004% 100.004% ( 380, 580 ] 11 0.001% 100.006% ( 580, 870 ] 17 0.002% 100.008% ( 870, 1300 ] 11 0.001% 100.009% ( 1300, 1900 ] 6 0.001% 100.010% ( 1900, 2900 ] 3 0.000% 100.010% ( 2900, 4400 ] 8 0.001% 100.011% ** Level 1 read latency histogram (micros): Count: 81485 Average: 2.5287 StdDev: 13.13 Min: 1 Median: 1.6577 Max: 2421 Percentiles: P50: 1.66 P75: 2.17 P99: 5.99 P99.9: 13.12 P99.99: 18.44 ------------------------------------------------------ [ 0, 1 ] 6235 7.652% 7.652% ## ( 1, 2 ] 52468 64.390% 72.041% ############# ( 2, 3 ] 13912 17.073% 89.115% ### ( 3, 4 ] 4462 5.476% 94.590% # ( 4, 6 ] 3616 4.438% 99.028% # ( 6, 10 ] 602 0.739% 99.767% ( 10, 15 ] 174 0.214% 99.980% ( 15, 22 ] 16 0.020% 100.000% ( 22, 34 ] 2 0.002% 100.002% ( 34, 51 ] 1 0.001% 100.004% ( 51, 76 ] 1 0.001% 100.005% ( 110, 170 ] 2 0.002% 100.007% ( 170, 250 ] 2 0.002% 100.010% ( 250, 380 ] 3 0.004% 100.013% ( 380, 580 ] 2 0.002% 100.016% ( 580, 870 ] 1 0.001% 100.017% ( 870, 1300 ] 1 0.001% 100.018% ( 1900, 2900 ] 2 0.002% 100.021% ** Level 2 read latency histogram (micros): Count: 95433 Average: 2.0099 StdDev: 5.91 Min: 1 Median: 1.3912 Max: 1118 Percentiles: P50: 1.39 P75: 1.79 P99: 5.52 P99.9: 10.73 P99.99: 27.83 ------------------------------------------------------ [ 0, 1 ] 24545 25.720% 25.720% ##### ( 1, 2 ] 59231 62.066% 87.785% ############ ( 2, 3 ] 6520 6.832% 94.617% # ( 3, 4 ] 2843 2.979% 97.596% # ( 4, 6 ] 1760 1.844% 99.440% ( 6, 10 ] 425 0.445% 99.886% ( 10, 15 ] 93 0.097% 99.983% ( 15, 22 ] 5 0.005% 99.988% ( 22, 34 ] 3 0.003% 99.992% ( 51, 76 ] 2 0.002% 99.994% ( 380, 580 ] 3 0.003% 99.997% ( 580, 870 ] 1 0.001% 99.998% ( 870, 1300 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57468 Average: 3.4841 StdDev: 46.68 Min: 0 Median: 1.6585 Max: 3622 Percentiles: P50: 1.66 P75: 2.19 P99: 6.85 P99.9: 40.00 P99.99: 2056.65 ------------------------------------------------------ [ 0, 1 ] 4626 8.050% 8.050% ## ( 1, 2 ] 36611 63.707% 71.756% ############# ( 2, 3 ] 9948 17.311% 89.067% ### ( 3, 4 ] 3121 5.431% 94.498% # ( 4, 6 ] 2493 4.338% 98.836% # ( 6, 10 ] 442 0.769% 99.605% ( 10, 15 ] 117 0.204% 99.809% ( 15, 22 ] 20 0.035% 99.843% ( 22, 34 ] 29 0.050% 99.894% ( 34, 51 ] 10 0.017% 99.911% ( 110, 170 ] 2 0.003% 99.915% ( 170, 250 ] 8 0.014% 99.929% ( 250, 380 ] 12 0.021% 99.950% ( 380, 580 ] 9 0.016% 99.965% ( 580, 870 ] 5 0.009% 99.974% ( 870, 1300 ] 4 0.007% 99.981% ( 1300, 1900 ] 4 0.007% 99.988% ( 1900, 2900 ] 8 0.014% 100.002% ( 2900, 4400 ] 5 0.009% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1664.2571 StdDev: 1044.76 Min: 510 Median: 1390.0000 Max: 4479 Percentiles: P50: 1390.00 P75: 1950.00 P99: 4479.00 P99.9: 4479.00 P99.99: 4479.00 ------------------------------------------------------ ( 380, 580 ] 3 8.571% 8.571% ## ( 580, 870 ] 5 14.286% 22.857% ### ( 870, 1300 ] 8 22.857% 45.714% ##### ( 1300, 1900 ] 10 28.571% 74.286% ###### ( 1900, 2900 ] 5 14.286% 88.571% ### ( 2900, 4400 ] 3 8.571% 97.143% ## ( 4400, 6600 ] 1 2.857% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 738.1000 StdDev: 827.14 Min: 97 Median: 380.0000 Max: 2698 Percentiles: P50: 380.00 P75: 870.00 P99: 2698.00 P99.9: 2698.00 P99.99: 2698.00 ------------------------------------------------------ ( 76, 110 ] 4 20.000% 20.000% #### ( 110, 170 ] 2 10.000% 30.000% ## ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 2 10.000% 50.000% ## ( 380, 580 ] 3 15.000% 65.000% ### ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 2 10.000% 85.000% ## ( 1900, 2900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 11.9 total, 2.9 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 331.66 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.419 H:M:S, 12.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 337.14 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.376 H:M:S, 12.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-06:42:10 ... thread 0: (1000000,5000000) ops and (128573.9,257035.8) ops/second in (7.777631,19.452544) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 4.12 GB 106.7 4.4 0.0 4.4 9.1 4.7 0.0 1.9 148.7 309.9 30.06 28.36 83 0.362 4540K 6712 0.0 0.0 L1 1/1 2.92 MB 0.0 0.4 0.4 0.0 0.4 0.4 0.2 1.0 238.4 238.0 1.59 1.54 1 1.586 383K 591 0.0 0.0 L2 2/0 55.35 MB 0.6 0.4 0.2 0.2 0.4 0.2 0.5 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 42/0 998.90 MB 1.0 0.8 0.6 0.2 0.8 0.6 0.1 1.4 204.6 204.4 3.89 3.45 3 1.296 808K 804 0.0 0.0 L4 144/0 9.53 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1255/15 100.87 GB 0.0 5.9 1.1 4.8 10.6 5.8 1.6 2.2 162.7 293.4 37.03 34.79 89 0.416 6116K 8304 0.0 0.0 Int 0/0 0.00 KB 0.0 5.9 1.1 4.8 10.6 5.8 1.5 2.2 162.7 293.4 37.03 34.79 89 0.416 6116K 8304 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.9 1.1 4.8 5.9 1.1 0.0 0.0 255.4 255.0 23.59 22.74 20 1.179 6116K 8304 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 4.7 4.7 0.0 0.0 0.0 360.7 13.45 12.05 69 0.195 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 19.6 total, 19.5 interval Flush(GB): cumulative 4.737, interval 4.737 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.61 GB write, 553.05 MB/s write, 5.88 GB read, 306.59 MB/s read, 37.0 seconds Interval compaction: 10.61 GB write, 558.55 MB/s write, 5.88 GB read, 309.64 MB/s read, 37.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 15, interval: 23 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1761329 Average: 2.0677 StdDev: 11.00 Min: 1 Median: 1.4112 Max: 3946 Percentiles: P50: 1.41 P75: 1.80 P99: 5.31 P99.9: 11.50 P99.99: 81.71 ------------------------------------------------------ [ 0, 1 ] 411469 23.361% 23.361% ##### ( 1, 2 ] 1141035 64.783% 88.144% ############# ( 2, 3 ] 130274 7.396% 95.540% # ( 3, 4 ] 46607 2.646% 98.186% # ( 4, 6 ] 21902 1.243% 99.430% ( 6, 10 ] 7804 0.443% 99.873% ( 10, 15 ] 1585 0.090% 99.963% ( 15, 22 ] 390 0.022% 99.985% ( 22, 34 ] 30 0.002% 99.987% ( 34, 51 ] 11 0.001% 99.987% ( 51, 76 ] 21 0.001% 99.989% ( 76, 110 ] 148 0.008% 99.997% ( 110, 170 ] 148 0.008% 100.005% ( 170, 250 ] 43 0.002% 100.008% ( 250, 380 ] 55 0.003% 100.011% ( 380, 580 ] 22 0.001% 100.012% ( 580, 870 ] 26 0.001% 100.014% ( 870, 1300 ] 18 0.001% 100.015% ( 1300, 1900 ] 9 0.001% 100.015% ( 1900, 2900 ] 7 0.000% 100.016% ( 2900, 4400 ] 9 0.001% 100.016% ** Level 1 read latency histogram (micros): Count: 81485 Average: 2.5287 StdDev: 13.13 Min: 1 Median: 1.6577 Max: 2421 Percentiles: P50: 1.66 P75: 2.17 P99: 5.99 P99.9: 13.12 P99.99: 18.44 ------------------------------------------------------ [ 0, 1 ] 6235 7.652% 7.652% ## ( 1, 2 ] 52468 64.390% 72.041% ############# ( 2, 3 ] 13912 17.073% 89.115% ### ( 3, 4 ] 4462 5.476% 94.590% # ( 4, 6 ] 3616 4.438% 99.028% # ( 6, 10 ] 602 0.739% 99.767% ( 10, 15 ] 174 0.214% 99.980% ( 15, 22 ] 16 0.020% 100.000% ( 22, 34 ] 2 0.002% 100.002% ( 34, 51 ] 1 0.001% 100.004% ( 51, 76 ] 1 0.001% 100.005% ( 110, 170 ] 2 0.002% 100.007% ( 170, 250 ] 2 0.002% 100.010% ( 250, 380 ] 3 0.004% 100.013% ( 380, 580 ] 2 0.002% 100.016% ( 580, 870 ] 1 0.001% 100.017% ( 870, 1300 ] 1 0.001% 100.018% ( 1900, 2900 ] 2 0.002% 100.021% ** Level 2 read latency histogram (micros): Count: 95433 Average: 2.0099 StdDev: 5.91 Min: 1 Median: 1.3912 Max: 1118 Percentiles: P50: 1.39 P75: 1.79 P99: 5.52 P99.9: 10.73 P99.99: 27.83 ------------------------------------------------------ [ 0, 1 ] 24545 25.720% 25.720% ##### ( 1, 2 ] 59231 62.066% 87.785% ############ ( 2, 3 ] 6520 6.832% 94.617% # ( 3, 4 ] 2843 2.979% 97.596% # ( 4, 6 ] 1760 1.844% 99.440% ( 6, 10 ] 425 0.445% 99.886% ( 10, 15 ] 93 0.097% 99.983% ( 15, 22 ] 5 0.005% 99.988% ( 22, 34 ] 3 0.003% 99.992% ( 51, 76 ] 2 0.002% 99.994% ( 380, 580 ] 3 0.003% 99.997% ( 580, 870 ] 1 0.001% 99.998% ( 870, 1300 ] 2 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 57468 Average: 3.4841 StdDev: 46.68 Min: 0 Median: 1.6585 Max: 3622 Percentiles: P50: 1.66 P75: 2.19 P99: 6.85 P99.9: 40.00 P99.99: 2056.65 ------------------------------------------------------ [ 0, 1 ] 4626 8.050% 8.050% ## ( 1, 2 ] 36611 63.707% 71.756% ############# ( 2, 3 ] 9948 17.311% 89.067% ### ( 3, 4 ] 3121 5.431% 94.498% # ( 4, 6 ] 2493 4.338% 98.836% # ( 6, 10 ] 442 0.769% 99.605% ( 10, 15 ] 117 0.204% 99.809% ( 15, 22 ] 20 0.035% 99.843% ( 22, 34 ] 29 0.050% 99.894% ( 34, 51 ] 10 0.017% 99.911% ( 110, 170 ] 2 0.003% 99.915% ( 170, 250 ] 8 0.014% 99.929% ( 250, 380 ] 12 0.021% 99.950% ( 380, 580 ] 9 0.016% 99.965% ( 580, 870 ] 5 0.009% 99.974% ( 870, 1300 ] 4 0.007% 99.981% ( 1300, 1900 ] 4 0.007% 99.988% ( 1900, 2900 ] 8 0.014% 100.002% ( 2900, 4400 ] 5 0.009% 100.010% ** Level 4 read latency histogram (micros): Count: 35 Average: 1664.2571 StdDev: 1044.76 Min: 510 Median: 1390.0000 Max: 4479 Percentiles: P50: 1390.00 P75: 1950.00 P99: 4479.00 P99.9: 4479.00 P99.99: 4479.00 ------------------------------------------------------ ( 380, 580 ] 3 8.571% 8.571% ## ( 580, 870 ] 5 14.286% 22.857% ### ( 870, 1300 ] 8 22.857% 45.714% ##### ( 1300, 1900 ] 10 28.571% 74.286% ###### ( 1900, 2900 ] 5 14.286% 88.571% ### ( 2900, 4400 ] 3 8.571% 97.143% ## ( 4400, 6600 ] 1 2.857% 100.000% # ** Level 5 read latency histogram (micros): Count: 20 Average: 738.1000 StdDev: 827.14 Min: 97 Median: 380.0000 Max: 2698 Percentiles: P50: 380.00 P75: 870.00 P99: 2698.00 P99.9: 2698.00 P99.99: 2698.00 ------------------------------------------------------ ( 76, 110 ] 4 20.000% 20.000% #### ( 110, 170 ] 2 10.000% 30.000% ## ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 2 10.000% 50.000% ## ( 380, 580 ] 3 15.000% 65.000% ### ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 2 10.000% 85.000% ## ( 1900, 2900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 19.6 total, 7.8 interval Cumulative writes: 0 writes, 5000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 4.81 GB, 250.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.556 H:M:S, 33.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 126.54 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.137 H:M:S, 66.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/07/15-06:42:19 ... thread 0: (1000000,6000000) ops and (121128.2,216541.9) ops/second in (8.255718,27.708262) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 6/4 1.96 GB 169.4 5.9 0.0 5.9 11.6 5.7 0.0 2.0 150.9 295.4 40.06 36.70 101 0.397 6139K 9947 0.0 0.0 L1 6/0 527.21 MB 52.7 3.4 3.4 0.0 3.4 3.4 0.2 1.0 221.2 218.2 15.95 15.01 2 7.975 3582K 47K 0.0 0.0 L2 5/5 517.02 MB 0.0 0.4 0.2 0.2 0.4 0.2 3.1 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 32/17 2.20 GB 1.1 1.5 1.1 0.4 1.5 1.1 1.6 1.3 151.5 151.1 9.94 6.70 8 1.242 1499K 2505 0.0 0.0 L4 164/6 10.37 GB 1.1 0.5 0.4 0.1 0.5 0.4 0.8 1.2 120.0 119.4 3.95 2.18 3 1.317 451K 1683 0.0 0.0 L5 1048/0 86.19 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1261/32 101.73 GB 0.0 11.7 5.1 6.5 17.2 10.7 6.0 3.0 167.1 247.4 71.40 62.03 116 0.615 12M 61K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.7 5.1 6.5 17.2 10.7 5.9 3.0 167.1 247.4 71.40 62.03 116 0.615 12M 61K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.7 5.1 6.5 11.6 5.1 0.0 0.0 219.2 218.0 54.44 47.55 33 1.650 12M 61K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 5.7 5.7 0.0 0.0 0.0 341.8 16.96 14.49 83 0.204 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 27.9 total, 27.7 interval Flush(GB): cumulative 5.660, interval 5.660 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.25 GB write, 633.03 MB/s write, 11.65 GB read, 427.60 MB/s read, 71.4 seconds Interval compaction: 17.25 GB write, 637.45 MB/s write, 11.65 GB read, 430.59 MB/s read, 71.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 27, interval: 43 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2493054 Average: 2.5053 StdDev: 19.12 Min: 1 Median: 1.4344 Max: 5611 Percentiles: P50: 1.43 P75: 1.82 P99: 6.24 P99.9: 129.33 P99.99: 312.14 ------------------------------------------------------ [ 0, 1 ] 548988 22.021% 22.021% #### ( 1, 2 ] 1605834 64.412% 86.433% ############# ( 2, 3 ] 197798 7.934% 94.367% ## ( 3, 4 ] 74656 2.995% 97.362% # ( 4, 6 ] 39830 1.598% 98.959% ( 6, 10 ] 16973 0.681% 99.640% ( 10, 15 ] 3044 0.122% 99.762% ( 15, 22 ] 767 0.031% 99.793% ( 22, 34 ] 47 0.002% 99.795% ( 34, 51 ] 36 0.001% 99.796% ( 51, 76 ] 84 0.003% 99.800% ( 76, 110 ] 1768 0.071% 99.870% ( 110, 170 ] 2284 0.092% 99.962% ( 170, 250 ] 526 0.021% 99.983% ( 250, 380 ] 355 0.014% 99.997% ( 380, 580 ] 282 0.011% 100.009% ( 580, 870 ] 138 0.006% 100.014% ( 870, 1300 ] 58 0.002% 100.017% ( 1300, 1900 ] 32 0.001% 100.018% ( 1900, 2900 ] 24 0.001% 100.019% ( 2900, 4400 ] 22 0.001% 100.020% ( 4400, 6600 ] 5 0.000% 100.020% ** Level 1 read latency histogram (micros): Count: 399152 Average: 5.5911 StdDev: 86.25 Min: 1 Median: 1.9404 Max: 8020 Percentiles: P50: 1.94 P75: 2.92 P99: 9.09 P99.9: 14.63 P99.99: 21.87 ------------------------------------------------------ [ 0, 1 ] 9648 2.417% 2.417% ( 1, 2 ] 201956 50.596% 53.013% ########## ( 2, 3 ] 94914 23.779% 76.792% ##### ( 3, 4 ] 41116 10.301% 87.093% ## ( 4, 6 ] 41804 10.473% 97.566% ## ( 6, 10 ] 7399 1.854% 99.420% ( 10, 15 ] 2068 0.518% 99.938% ( 15, 22 ] 211 0.053% 99.991% ( 22, 34 ] 14 0.004% 99.994% ( 34, 51 ] 8 0.002% 99.996% ( 51, 76 ] 15 0.004% 100.000% ( 76, 110 ] 70 0.018% 100.018% ( 110, 170 ] 99 0.025% 100.043% ( 170, 250 ] 54 0.014% 100.056% ( 250, 380 ] 73 0.018% 100.074% ( 380, 580 ] 112 0.028% 100.102% ( 580, 870 ] 139 0.035% 100.137% ( 870, 1300 ] 81 0.020% 100.158% ( 1300, 1900 ] 49 0.012% 100.170% ( 1900, 2900 ] 78 0.020% 100.189% ( 2900, 4400 ] 92 0.023% 100.212% ( 4400, 6600 ] 32 0.008% 100.220% ( 6600, 9900 ] 4 0.001% 100.221% ** Level 2 read latency histogram (micros): Count: 95563 Average: 3.2803 StdDev: 62.70 Min: 1 Median: 1.3922 Max: 5920 Percentiles: P50: 1.39 P75: 1.80 P99: 5.60 P99.9: 14.16 P99.99: 3920.69 ------------------------------------------------------ [ 0, 1 ] 24545 25.685% 25.685% ##### ( 1, 2 ] 59241 61.992% 87.676% ############ ( 2, 3 ] 6554 6.858% 94.534% # ( 3, 4 ] 2851 2.983% 97.518% # ( 4, 6 ] 1771 1.853% 99.371% ( 6, 10 ] 428 0.448% 99.819% ( 10, 15 ] 93 0.097% 99.916% ( 15, 22 ] 5 0.005% 99.922% ( 22, 34 ] 3 0.003% 99.925% ( 51, 76 ] 2 0.002% 99.927% ( 250, 380 ] 5 0.005% 99.932% ( 380, 580 ] 11 0.012% 99.943% ( 580, 870 ] 7 0.007% 99.951% ( 870, 1300 ] 16 0.017% 99.968% ( 1300, 1900 ] 8 0.008% 99.976% ( 1900, 2900 ] 8 0.008% 99.984% ( 2900, 4400 ] 8 0.008% 99.993% ( 4400, 6600 ] 7 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 203354 Average: 17.8215 StdDev: 188.99 Min: 0 Median: 1.8689 Max: 9391 Percentiles: P50: 1.87 P75: 2.85 P99: 295.17 P99.9: 2509.56 P99.99: 3665.39 ------------------------------------------------------ [ 0, 1 ] 7801 3.836% 3.836% # ( 1, 2 ] 108036 53.127% 56.963% ########### ( 2, 3 ] 43074 21.182% 78.145% #### ( 3, 4 ] 20062 9.866% 88.011% ## ( 4, 6 ] 16859 8.290% 96.301% ## ( 6, 10 ] 3885 1.910% 98.211% ( 10, 15 ] 990 0.487% 98.698% ( 15, 22 ] 124 0.061% 98.759% ( 22, 34 ] 45 0.022% 98.781% ( 34, 51 ] 16 0.008% 98.789% ( 51, 76 ] 4 0.002% 98.791% ( 76, 110 ] 33 0.016% 98.807% ( 110, 170 ] 133 0.065% 98.873% ( 170, 250 ] 165 0.081% 98.954% ( 250, 380 ] 269 0.132% 99.086% ( 380, 580 ] 421 0.207% 99.293% ( 580, 870 ] 498 0.245% 99.538% ( 870, 1300 ] 391 0.192% 99.731% ( 1300, 1900 ] 227 0.112% 99.842% ( 1900, 2900 ] 193 0.095% 99.937% ( 2900, 4400 ] 211 0.104% 100.041% ( 4400, 6600 ] 67 0.033% 100.074% ( 6600, 9900 ] 7 0.003% 100.077% ** Level 4 read latency histogram (micros): Count: 85 Average: 832.6471 StdDev: 1036.53 Min: 1 Median: 480.0000 Max: 4479 Percentiles: P50: 480.00 P75: 1380.77 P99: 4479.00 P99.9: 4479.00 P99.99: 4479.00 ------------------------------------------------------ [ 0, 1 ] 7 8.235% 8.235% ## ( 1, 2 ] 20 23.529% 31.765% ##### ( 2, 3 ] 4 4.706% 36.471% # ( 4, 6 ] 1 1.176% 37.647% ( 110, 170 ] 3 3.529% 41.176% # ( 170, 250 ] 6 7.059% 48.235% # ( 380, 580 ] 3 3.529% 51.765% # ( 580, 870 ] 8 9.412% 61.176% ## ( 870, 1300 ] 10 11.765% 72.941% ## ( 1300, 1900 ] 13 15.294% 88.235% ### ( 1900, 2900 ] 6 7.059% 95.294% # ( 2900, 4400 ] 3 3.529% 98.824% # ( 4400, 6600 ] 1 1.176% 100.000% ** Level 5 read latency histogram (micros): Count: 20 Average: 738.1000 StdDev: 827.14 Min: 97 Median: 380.0000 Max: 2698 Percentiles: P50: 380.00 P75: 870.00 P99: 2698.00 P99.9: 2698.00 P99.99: 2698.00 ------------------------------------------------------ ( 76, 110 ] 4 20.000% 20.000% #### ( 110, 170 ] 2 10.000% 30.000% ## ( 170, 250 ] 2 10.000% 40.000% ## ( 250, 380 ] 2 10.000% 50.000% ## ( 380, 580 ] 3 15.000% 65.000% ### ( 580, 870 ] 2 10.000% 75.000% ## ( 1300, 1900 ] 2 10.000% 85.000% ## ( 1900, 2900 ] 3 15.000% 100.000% ### ** DB Stats ** Uptime(secs): 27.9 total, 8.3 interval Cumulative writes: 0 writes, 6000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 5.77 GB, 211.63 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:12.018 H:M:S, 43.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 119.21 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.462 H:M:S, 66.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/07/15-06:42:23 ... thread 0: (1000000,7000000) ops and (259550.3,221792.2) ops/second in (3.852818,31.561080) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/9 2.95 GB 56.7 6.5 0.0 6.5 13.1 6.6 0.0 2.0 143.1 290.1 46.22 41.55 116 0.398 6714K 10K 0.0 0.0 L1 6/6 527.21 MB 0.0 3.4 3.4 0.0 3.4 3.4 0.2 1.0 221.2 218.2 15.95 15.01 2 7.975 3582K 47K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 3.1 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 13/0 873.05 MB 0.9 2.8 1.6 1.2 2.8 1.6 1.6 1.7 153.1 152.8 18.94 12.70 13 1.457 2916K 5007 0.0 0.0 L4 162/0 9.57 GB 1.0 1.8 1.4 0.4 1.8 1.4 1.6 1.3 135.2 134.6 13.39 8.38 18 0.744 1779K 6620 0.0 0.0 L5 1080/0 88.83 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1274/15 102.71 GB 0.0 14.9 6.6 8.2 21.4 13.2 9.5 3.2 158.6 228.8 96.00 79.09 151 0.636 15M 69K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.9 6.6 8.2 21.4 13.2 9.4 3.2 158.6 228.8 96.00 79.09 151 0.636 15M 69K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.9 6.6 8.2 14.8 6.6 0.0 0.0 202.0 201.0 75.39 61.99 55 1.371 15M 69K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 6.6 6.6 0.0 0.0 0.0 330.2 20.60 17.10 96 0.215 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 31.8 total, 31.6 interval Flush(GB): cumulative 6.644, interval 6.644 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 21.45 GB write, 691.53 MB/s write, 14.87 GB read, 479.57 MB/s read, 96.0 seconds Interval compaction: 21.45 GB write, 695.77 MB/s write, 14.87 GB read, 482.51 MB/s read, 96.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 43, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 43, interval: 59 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 2821028 Average: 2.5411 StdDev: 19.57 Min: 1 Median: 1.4603 Max: 5611 Percentiles: P50: 1.46 P75: 1.85 P99: 6.50 P99.9: 124.37 P99.99: 308.99 ------------------------------------------------------ [ 0, 1 ] 575652 20.406% 20.406% #### ( 1, 2 ] 1813840 64.297% 84.703% ############# ( 2, 3 ] 248990 8.826% 93.529% ## ( 3, 4 ] 94804 3.361% 96.890% # ( 4, 6 ] 56897 2.017% 98.907% ( 6, 10 ] 20961 0.743% 99.650% ( 10, 15 ] 3697 0.131% 99.781% ( 15, 22 ] 818 0.029% 99.810% ( 22, 34 ] 56 0.002% 99.812% ( 34, 51 ] 41 0.001% 99.813% ( 51, 76 ] 87 0.003% 99.816% ( 76, 110 ] 1801 0.064% 99.880% ( 110, 170 ] 2363 0.084% 99.964% ( 170, 250 ] 569 0.020% 99.984% ( 250, 380 ] 381 0.014% 99.997% ( 380, 580 ] 315 0.011% 100.009% ( 580, 870 ] 156 0.006% 100.014% ( 870, 1300 ] 68 0.002% 100.017% ( 1300, 1900 ] 35 0.001% 100.018% ( 1900, 2900 ] 29 0.001% 100.019% ( 2900, 4400 ] 29 0.001% 100.020% ( 4400, 6600 ] 6 0.000% 100.020% ** Level 1 read latency histogram (micros): Count: 579751 Average: 6.5410 StdDev: 107.01 Min: 1 Median: 1.9366 Max: 10767 Percentiles: P50: 1.94 P75: 2.92 P99: 9.40 P99.9: 19.60 P99.99: 279.46 ------------------------------------------------------ [ 0, 1 ] 13857 2.390% 2.390% ( 1, 2 ] 294696 50.831% 53.222% ########## ( 2, 3 ] 136886 23.611% 76.833% ##### ( 3, 4 ] 58934 10.165% 86.998% ## ( 4, 6 ] 59484 10.260% 97.258% ## ( 6, 10 ] 11870 2.047% 99.306% ( 10, 15 ] 3213 0.554% 99.860% ( 15, 22 ] 352 0.061% 99.921% ( 22, 34 ] 19 0.003% 99.924% ( 34, 51 ] 12 0.002% 99.926% ( 51, 76 ] 16 0.003% 99.929% ( 76, 110 ] 77 0.013% 99.942% ( 110, 170 ] 161 0.028% 99.970% ( 170, 250 ] 92 0.016% 99.986% ( 250, 380 ] 106 0.018% 100.004% ( 380, 580 ] 164 0.028% 100.032% ( 580, 870 ] 189 0.033% 100.065% ( 870, 1300 ] 130 0.022% 100.087% ( 1300, 1900 ] 89 0.015% 100.103% ( 1900, 2900 ] 170 0.029% 100.132% ( 2900, 4400 ] 187 0.032% 100.164% ( 4400, 6600 ] 62 0.011% 100.175% ( 6600, 9900 ] 19 0.003% 100.178% ( 9900, 14000 ] 1 0.000% 100.179% ** Level 2 read latency histogram (micros): Count: 95914 Average: 3.4081 StdDev: 63.41 Min: 1 Median: 1.3943 Max: 5920 Percentiles: P50: 1.39 P75: 1.80 P99: 5.65 P99.9: 155.43 P99.99: 3914.11 ------------------------------------------------------ [ 0, 1 ] 24550 25.596% 25.596% ##### ( 1, 2 ] 59364 61.893% 87.489% ############ ( 2, 3 ] 6689 6.974% 94.463% # ( 3, 4 ] 2879 3.002% 97.464% # ( 4, 6 ] 1786 1.862% 99.326% ( 6, 10 ] 434 0.452% 99.779% ( 10, 15 ] 94 0.098% 99.877% ( 15, 22 ] 5 0.005% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 51, 76 ] 2 0.002% 99.887% ( 76, 110 ] 3 0.003% 99.891% ( 110, 170 ] 12 0.013% 99.903% ( 170, 250 ] 9 0.009% 99.912% ( 250, 380 ] 10 0.010% 99.923% ( 380, 580 ] 15 0.016% 99.938% ( 580, 870 ] 11 0.011% 99.950% ( 870, 1300 ] 16 0.017% 99.967% ( 1300, 1900 ] 8 0.008% 99.975% ( 1900, 2900 ] 9 0.009% 99.984% ( 2900, 4400 ] 8 0.008% 99.993% ( 4400, 6600 ] 7 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 526917 Average: 12.1714 StdDev: 151.73 Min: 0 Median: 1.8849 Max: 9391 Percentiles: P50: 1.88 P75: 2.87 P99: 11.93 P99.9: 1248.76 P99.99: 2157.73 ------------------------------------------------------ [ 0, 1 ] 17701 3.359% 3.359% # ( 1, 2 ] 277716 52.706% 56.065% ########### ( 2, 3 ] 114267 21.686% 77.751% #### ( 3, 4 ] 53184 10.093% 87.845% ## ( 4, 6 ] 48053 9.120% 96.964% ## ( 6, 10 ] 9988 1.896% 98.860% ( 10, 15 ] 1915 0.363% 99.223% ( 15, 22 ] 290 0.055% 99.278% ( 22, 34 ] 80 0.015% 99.293% ( 34, 51 ] 33 0.006% 99.300% ( 51, 76 ] 11 0.002% 99.302% ( 76, 110 ] 82 0.016% 99.317% ( 110, 170 ] 286 0.054% 99.372% ( 170, 250 ] 339 0.064% 99.436% ( 250, 380 ] 483 0.092% 99.528% ( 380, 580 ] 673 0.128% 99.655% ( 580, 870 ] 787 0.149% 99.805% ( 870, 1300 ] 570 0.108% 99.913% ( 1300, 1900 ] 321 0.061% 99.974% ( 1900, 2900 ] 331 0.063% 100.037% ( 2900, 4400 ] 338 0.064% 100.101% ( 4400, 6600 ] 106 0.020% 100.121% ( 6600, 9900 ] 19 0.004% 100.124% ** Level 4 read latency histogram (micros): Count: 23471 Average: 7.3076 StdDev: 91.50 Min: 0 Median: 1.9428 Max: 4479 Percentiles: P50: 1.94 P75: 2.92 P99: 9.61 P99.9: 1481.16 P99.99: 3894.84 ------------------------------------------------------ [ 0, 1 ] 211 0.899% 0.899% ( 1, 2 ] 12224 52.081% 52.980% ########## ( 2, 3 ] 5630 23.987% 76.967% ##### ( 3, 4 ] 2459 10.477% 87.444% ## ( 4, 6 ] 2440 10.396% 97.840% ## ( 6, 10 ] 302 1.287% 99.127% ( 10, 15 ] 86 0.366% 99.493% ( 15, 22 ] 17 0.072% 99.565% ( 76, 110 ] 2 0.009% 99.574% ( 110, 170 ] 7 0.030% 99.604% ( 170, 250 ] 25 0.107% 99.710% ( 250, 380 ] 1 0.004% 99.715% ( 380, 580 ] 4 0.017% 99.732% ( 580, 870 ] 15 0.064% 99.795% ( 870, 1300 ] 20 0.085% 99.881% ( 1300, 1900 ] 15 0.064% 99.945% ( 1900, 2900 ] 8 0.034% 99.979% ( 2900, 4400 ] 4 0.017% 99.996% ( 4400, 6600 ] 1 0.004% 100.000% ** Level 5 read latency histogram (micros): Count: 180 Average: 1466.1056 StdDev: 1511.67 Min: 85 Median: 826.5000 Max: 6993 Percentiles: P50: 826.50 P75: 2344.44 P99: 6930.00 P99.9: 6993.00 P99.99: 6993.00 ------------------------------------------------------ ( 76, 110 ] 6 3.333% 3.333% # ( 110, 170 ] 12 6.667% 10.000% # ( 170, 250 ] 15 8.333% 18.333% ## ( 250, 380 ] 19 10.556% 28.889% ## ( 380, 580 ] 21 11.667% 40.556% ## ( 580, 870 ] 20 11.111% 51.667% ## ( 870, 1300 ] 22 12.222% 63.889% ## ( 1300, 1900 ] 12 6.667% 70.556% # ( 1900, 2900 ] 18 10.000% 80.556% ## ( 2900, 4400 ] 21 11.667% 92.222% ## ( 4400, 6600 ] 12 6.667% 98.889% # ( 6600, 9900 ] 2 1.111% 100.000% ** DB Stats ** Uptime(secs): 31.8 total, 3.9 interval Cumulative writes: 0 writes, 7000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 6.73 GB, 216.94 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:13.218 H:M:S, 41.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 255.44 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.200 H:M:S, 31.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-06:42:26 ... thread 0: (1000000,8000000) ops and (334399.2,231538.3) ops/second in (2.990438,34.551518) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/9 3.87 GB 144.8 7.2 0.0 7.2 14.8 7.6 0.0 1.9 142.5 292.2 51.70 46.70 132 0.392 7482K 11K 0.0 0.0 L1 6/6 527.21 MB 0.0 3.4 3.4 0.0 3.4 3.4 0.2 1.0 221.2 218.2 15.95 15.01 2 7.975 3582K 47K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 3.1 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 13/0 873.05 MB 0.9 2.8 1.6 1.2 2.8 1.6 1.6 1.7 153.1 152.8 18.94 12.70 13 1.457 2916K 5007 0.0 0.0 L4 162/0 9.57 GB 1.0 1.8 1.4 0.4 1.8 1.4 1.6 1.3 135.2 134.6 13.39 8.38 18 0.744 1779K 6620 0.0 0.0 L5 1080/0 88.83 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1278/15 103.63 GB 0.0 15.6 6.6 9.0 23.1 14.1 9.5 3.1 157.5 233.2 101.48 84.24 167 0.608 16M 70K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.6 6.6 9.0 23.1 14.1 9.4 3.1 157.5 233.2 101.48 84.24 167 0.608 16M 70K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.6 6.6 9.0 15.5 6.6 0.0 0.0 204.4 203.5 78.19 64.73 58 1.348 16M 70K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 7.6 7.6 0.0 0.0 0.0 332.7 23.29 19.50 109 0.214 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 34.7 total, 34.6 interval Flush(GB): cumulative 7.567, interval 7.567 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.11 GB write, 680.95 MB/s write, 15.61 GB read, 460.05 MB/s read, 101.5 seconds Interval compaction: 23.11 GB write, 684.76 MB/s write, 15.61 GB read, 462.63 MB/s read, 101.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 16, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 16, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 46, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 16, total-stops: 46, interval: 62 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3162984 Average: 2.5087 StdDev: 18.91 Min: 1 Median: 1.4635 Max: 5611 Percentiles: P50: 1.46 P75: 1.85 P99: 6.49 P99.9: 115.83 P99.99: 278.64 ------------------------------------------------------ [ 0, 1 ] 633061 20.015% 20.015% #### ( 1, 2 ] 2046115 64.689% 84.704% ############# ( 2, 3 ] 282539 8.933% 93.637% ## ( 3, 4 ] 104598 3.307% 96.944% # ( 4, 6 ] 62083 1.963% 98.906% ( 6, 10 ] 24111 0.762% 99.669% ( 10, 15 ] 4165 0.132% 99.800% ( 15, 22 ] 848 0.027% 99.827% ( 22, 34 ] 58 0.002% 99.829% ( 34, 51 ] 42 0.001% 99.830% ( 51, 76 ] 108 0.003% 99.834% ( 76, 110 ] 1859 0.059% 99.893% ( 110, 170 ] 2418 0.076% 99.969% ( 170, 250 ] 578 0.018% 99.987% ( 250, 380 ] 389 0.012% 100.000% ( 380, 580 ] 329 0.010% 100.010% ( 580, 870 ] 163 0.005% 100.015% ( 870, 1300 ] 69 0.002% 100.017% ( 1300, 1900 ] 37 0.001% 100.019% ( 1900, 2900 ] 31 0.001% 100.020% ( 2900, 4400 ] 31 0.001% 100.020% ( 4400, 6600 ] 6 0.000% 100.021% ** Level 1 read latency histogram (micros): Count: 579751 Average: 6.5410 StdDev: 107.01 Min: 1 Median: 1.9366 Max: 10767 Percentiles: P50: 1.94 P75: 2.92 P99: 9.40 P99.9: 19.60 P99.99: 279.46 ------------------------------------------------------ [ 0, 1 ] 13857 2.390% 2.390% ( 1, 2 ] 294696 50.831% 53.222% ########## ( 2, 3 ] 136886 23.611% 76.833% ##### ( 3, 4 ] 58934 10.165% 86.998% ## ( 4, 6 ] 59484 10.260% 97.258% ## ( 6, 10 ] 11870 2.047% 99.306% ( 10, 15 ] 3213 0.554% 99.860% ( 15, 22 ] 352 0.061% 99.921% ( 22, 34 ] 19 0.003% 99.924% ( 34, 51 ] 12 0.002% 99.926% ( 51, 76 ] 16 0.003% 99.929% ( 76, 110 ] 77 0.013% 99.942% ( 110, 170 ] 161 0.028% 99.970% ( 170, 250 ] 92 0.016% 99.986% ( 250, 380 ] 106 0.018% 100.004% ( 380, 580 ] 164 0.028% 100.032% ( 580, 870 ] 189 0.033% 100.065% ( 870, 1300 ] 130 0.022% 100.087% ( 1300, 1900 ] 89 0.015% 100.103% ( 1900, 2900 ] 170 0.029% 100.132% ( 2900, 4400 ] 187 0.032% 100.164% ( 4400, 6600 ] 62 0.011% 100.175% ( 6600, 9900 ] 19 0.003% 100.178% ( 9900, 14000 ] 1 0.000% 100.179% ** Level 2 read latency histogram (micros): Count: 95914 Average: 3.4081 StdDev: 63.41 Min: 1 Median: 1.3943 Max: 5920 Percentiles: P50: 1.39 P75: 1.80 P99: 5.65 P99.9: 155.43 P99.99: 3914.11 ------------------------------------------------------ [ 0, 1 ] 24550 25.596% 25.596% ##### ( 1, 2 ] 59364 61.893% 87.489% ############ ( 2, 3 ] 6689 6.974% 94.463% # ( 3, 4 ] 2879 3.002% 97.464% # ( 4, 6 ] 1786 1.862% 99.326% ( 6, 10 ] 434 0.452% 99.779% ( 10, 15 ] 94 0.098% 99.877% ( 15, 22 ] 5 0.005% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 51, 76 ] 2 0.002% 99.887% ( 76, 110 ] 3 0.003% 99.891% ( 110, 170 ] 12 0.013% 99.903% ( 170, 250 ] 9 0.009% 99.912% ( 250, 380 ] 10 0.010% 99.923% ( 380, 580 ] 15 0.016% 99.938% ( 580, 870 ] 11 0.011% 99.950% ( 870, 1300 ] 16 0.017% 99.967% ( 1300, 1900 ] 8 0.008% 99.975% ( 1900, 2900 ] 9 0.009% 99.984% ( 2900, 4400 ] 8 0.008% 99.993% ( 4400, 6600 ] 7 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 526917 Average: 12.1714 StdDev: 151.73 Min: 0 Median: 1.8849 Max: 9391 Percentiles: P50: 1.88 P75: 2.87 P99: 11.93 P99.9: 1248.76 P99.99: 2157.73 ------------------------------------------------------ [ 0, 1 ] 17701 3.359% 3.359% # ( 1, 2 ] 277716 52.706% 56.065% ########### ( 2, 3 ] 114267 21.686% 77.751% #### ( 3, 4 ] 53184 10.093% 87.845% ## ( 4, 6 ] 48053 9.120% 96.964% ## ( 6, 10 ] 9988 1.896% 98.860% ( 10, 15 ] 1915 0.363% 99.223% ( 15, 22 ] 290 0.055% 99.278% ( 22, 34 ] 80 0.015% 99.293% ( 34, 51 ] 33 0.006% 99.300% ( 51, 76 ] 11 0.002% 99.302% ( 76, 110 ] 82 0.016% 99.317% ( 110, 170 ] 286 0.054% 99.372% ( 170, 250 ] 339 0.064% 99.436% ( 250, 380 ] 483 0.092% 99.528% ( 380, 580 ] 673 0.128% 99.655% ( 580, 870 ] 787 0.149% 99.805% ( 870, 1300 ] 570 0.108% 99.913% ( 1300, 1900 ] 321 0.061% 99.974% ( 1900, 2900 ] 331 0.063% 100.037% ( 2900, 4400 ] 338 0.064% 100.101% ( 4400, 6600 ] 106 0.020% 100.121% ( 6600, 9900 ] 19 0.004% 100.124% ** Level 4 read latency histogram (micros): Count: 23471 Average: 7.3076 StdDev: 91.50 Min: 0 Median: 1.9428 Max: 4479 Percentiles: P50: 1.94 P75: 2.92 P99: 9.61 P99.9: 1481.16 P99.99: 3894.84 ------------------------------------------------------ [ 0, 1 ] 211 0.899% 0.899% ( 1, 2 ] 12224 52.081% 52.980% ########## ( 2, 3 ] 5630 23.987% 76.967% ##### ( 3, 4 ] 2459 10.477% 87.444% ## ( 4, 6 ] 2440 10.396% 97.840% ## ( 6, 10 ] 302 1.287% 99.127% ( 10, 15 ] 86 0.366% 99.493% ( 15, 22 ] 17 0.072% 99.565% ( 76, 110 ] 2 0.009% 99.574% ( 110, 170 ] 7 0.030% 99.604% ( 170, 250 ] 25 0.107% 99.710% ( 250, 380 ] 1 0.004% 99.715% ( 380, 580 ] 4 0.017% 99.732% ( 580, 870 ] 15 0.064% 99.795% ( 870, 1300 ] 20 0.085% 99.881% ( 1300, 1900 ] 15 0.064% 99.945% ( 1900, 2900 ] 8 0.034% 99.979% ( 2900, 4400 ] 4 0.017% 99.996% ( 4400, 6600 ] 1 0.004% 100.000% ** Level 5 read latency histogram (micros): Count: 180 Average: 1466.1056 StdDev: 1511.67 Min: 85 Median: 826.5000 Max: 6993 Percentiles: P50: 826.50 P75: 2344.44 P99: 6930.00 P99.9: 6993.00 P99.99: 6993.00 ------------------------------------------------------ ( 76, 110 ] 6 3.333% 3.333% # ( 110, 170 ] 12 6.667% 10.000% # ( 170, 250 ] 15 8.333% 18.333% ## ( 250, 380 ] 19 10.556% 28.889% ## ( 380, 580 ] 21 11.667% 40.556% ## ( 580, 870 ] 20 11.111% 51.667% ## ( 870, 1300 ] 22 12.222% 63.889% ## ( 1300, 1900 ] 12 6.667% 70.556% # ( 1900, 2900 ] 18 10.000% 80.556% ## ( 2900, 4400 ] 21 11.667% 92.222% ## ( 4400, 6600 ] 12 6.667% 98.889% # ( 6600, 9900 ] 2 1.111% 100.000% ** DB Stats ** Uptime(secs): 34.7 total, 3.0 interval Cumulative writes: 0 writes, 8000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 7.69 GB, 226.60 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:13.659 H:M:S, 39.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 329.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.440 H:M:S, 14.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-06:42:31 ... thread 0: (1000000,9000000) ops and (184744.0,225200.4) ops/second in (5.412895,39.964413) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 4.91 GB 201.4 8.6 0.0 8.6 17.2 8.6 0.0 2.0 145.9 292.7 60.01 54.57 153 0.392 8889K 12K 0.0 0.0 L1 6/6 527.21 MB 0.0 3.4 3.4 0.0 3.4 3.4 0.2 1.0 221.2 218.2 15.95 15.01 2 7.975 3582K 47K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 3.1 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 13/0 873.05 MB 0.9 2.8 1.6 1.2 2.8 1.6 1.6 1.7 153.1 152.8 18.94 12.70 13 1.457 2916K 5007 0.0 0.0 L4 162/0 9.57 GB 1.0 1.8 1.4 0.4 1.8 1.4 1.6 1.3 135.2 134.6 13.39 8.38 18 0.744 1779K 6620 0.0 0.0 L5 1080/0 88.83 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1279/15 104.68 GB 0.0 17.0 6.6 10.3 25.5 15.2 9.5 3.0 158.2 237.9 109.78 92.11 188 0.584 17M 71K 0.0 0.0 Int 0/0 0.00 KB 0.0 17.0 6.6 10.3 25.5 15.2 9.4 3.0 158.2 237.9 109.78 92.11 188 0.584 17M 71K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 17.0 6.6 10.3 16.9 6.6 0.0 0.0 208.3 207.4 83.41 69.88 63 1.324 17M 71K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 8.6 8.6 0.0 0.0 0.0 334.4 26.37 22.23 125 0.211 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 40.2 total, 40.0 interval Flush(GB): cumulative 8.613, interval 8.613 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 25.50 GB write, 650.31 MB/s write, 16.96 GB read, 432.55 MB/s read, 109.8 seconds Interval compaction: 25.50 GB write, 653.45 MB/s write, 16.96 GB read, 434.65 MB/s read, 109.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 26, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 26, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 48, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 26, total-stops: 48, interval: 74 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 3716214 Average: 2.4662 StdDev: 17.70 Min: 1 Median: 1.4696 Max: 5611 Percentiles: P50: 1.47 P75: 1.85 P99: 6.37 P99.9: 104.33 P99.99: 240.18 ------------------------------------------------------ [ 0, 1 ] 719592 19.364% 19.364% #### ( 1, 2 ] 2424438 65.239% 84.603% ############# ( 2, 3 ] 339221 9.128% 93.731% ## ( 3, 4 ] 121872 3.279% 97.011% # ( 4, 6 ] 71300 1.919% 98.929% ( 6, 10 ] 28490 0.767% 99.696% ( 10, 15 ] 4851 0.131% 99.826% ( 15, 22 ] 914 0.025% 99.851% ( 22, 34 ] 60 0.002% 99.853% ( 34, 51 ] 47 0.001% 99.854% ( 51, 76 ] 115 0.003% 99.857% ( 76, 110 ] 1920 0.052% 99.909% ( 110, 170 ] 2498 0.067% 99.976% ( 170, 250 ] 600 0.016% 99.992% ( 250, 380 ] 409 0.011% 100.003% ( 380, 580 ] 345 0.009% 100.012% ( 580, 870 ] 174 0.005% 100.017% ( 870, 1300 ] 76 0.002% 100.019% ( 1300, 1900 ] 38 0.001% 100.020% ( 1900, 2900 ] 33 0.001% 100.021% ( 2900, 4400 ] 31 0.001% 100.022% ( 4400, 6600 ] 6 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 679243 Average: 5.9480 StdDev: 98.96 Min: 1 Median: 1.8495 Max: 10767 Percentiles: P50: 1.85 P75: 2.80 P99: 9.29 P99.9: 55.76 P99.99: 298.97 ------------------------------------------------------ [ 0, 1 ] 32967 4.853% 4.853% # ( 1, 2 ] 360981 53.145% 57.998% ########### ( 2, 3 ] 145263 21.386% 79.384% #### ( 3, 4 ] 61363 9.034% 88.418% ## ( 4, 6 ] 60884 8.964% 97.382% ## ( 6, 10 ] 13373 1.969% 99.350% ( 10, 15 ] 3331 0.490% 99.841% ( 15, 22 ] 366 0.054% 99.895% ( 22, 34 ] 20 0.003% 99.898% ( 34, 51 ] 12 0.002% 99.899% ( 51, 76 ] 25 0.004% 99.903% ( 76, 110 ] 198 0.029% 99.932% ( 110, 170 ] 233 0.034% 99.967% ( 170, 250 ] 116 0.017% 99.984% ( 250, 380 ] 117 0.017% 100.001% ( 380, 580 ] 176 0.026% 100.027% ( 580, 870 ] 194 0.029% 100.055% ( 870, 1300 ] 132 0.019% 100.075% ( 1300, 1900 ] 89 0.013% 100.088% ( 1900, 2900 ] 170 0.025% 100.113% ( 2900, 4400 ] 187 0.028% 100.140% ( 4400, 6600 ] 62 0.009% 100.150% ( 6600, 9900 ] 19 0.003% 100.152% ( 9900, 14000 ] 1 0.000% 100.153% ** Level 2 read latency histogram (micros): Count: 95914 Average: 3.4081 StdDev: 63.41 Min: 1 Median: 1.3943 Max: 5920 Percentiles: P50: 1.39 P75: 1.80 P99: 5.65 P99.9: 155.43 P99.99: 3914.11 ------------------------------------------------------ [ 0, 1 ] 24550 25.596% 25.596% ##### ( 1, 2 ] 59364 61.893% 87.489% ############ ( 2, 3 ] 6689 6.974% 94.463% # ( 3, 4 ] 2879 3.002% 97.464% # ( 4, 6 ] 1786 1.862% 99.326% ( 6, 10 ] 434 0.452% 99.779% ( 10, 15 ] 94 0.098% 99.877% ( 15, 22 ] 5 0.005% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 51, 76 ] 2 0.002% 99.887% ( 76, 110 ] 3 0.003% 99.891% ( 110, 170 ] 12 0.013% 99.903% ( 170, 250 ] 9 0.009% 99.912% ( 250, 380 ] 10 0.010% 99.923% ( 380, 580 ] 15 0.016% 99.938% ( 580, 870 ] 11 0.011% 99.950% ( 870, 1300 ] 16 0.017% 99.967% ( 1300, 1900 ] 8 0.008% 99.975% ( 1900, 2900 ] 9 0.009% 99.984% ( 2900, 4400 ] 8 0.008% 99.993% ( 4400, 6600 ] 7 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 526917 Average: 12.1714 StdDev: 151.73 Min: 0 Median: 1.8849 Max: 9391 Percentiles: P50: 1.88 P75: 2.87 P99: 11.93 P99.9: 1248.76 P99.99: 2157.73 ------------------------------------------------------ [ 0, 1 ] 17701 3.359% 3.359% # ( 1, 2 ] 277716 52.706% 56.065% ########### ( 2, 3 ] 114267 21.686% 77.751% #### ( 3, 4 ] 53184 10.093% 87.845% ## ( 4, 6 ] 48053 9.120% 96.964% ## ( 6, 10 ] 9988 1.896% 98.860% ( 10, 15 ] 1915 0.363% 99.223% ( 15, 22 ] 290 0.055% 99.278% ( 22, 34 ] 80 0.015% 99.293% ( 34, 51 ] 33 0.006% 99.300% ( 51, 76 ] 11 0.002% 99.302% ( 76, 110 ] 82 0.016% 99.317% ( 110, 170 ] 286 0.054% 99.372% ( 170, 250 ] 339 0.064% 99.436% ( 250, 380 ] 483 0.092% 99.528% ( 380, 580 ] 673 0.128% 99.655% ( 580, 870 ] 787 0.149% 99.805% ( 870, 1300 ] 570 0.108% 99.913% ( 1300, 1900 ] 321 0.061% 99.974% ( 1900, 2900 ] 331 0.063% 100.037% ( 2900, 4400 ] 338 0.064% 100.101% ( 4400, 6600 ] 106 0.020% 100.121% ( 6600, 9900 ] 19 0.004% 100.124% ** Level 4 read latency histogram (micros): Count: 23471 Average: 7.3076 StdDev: 91.50 Min: 0 Median: 1.9428 Max: 4479 Percentiles: P50: 1.94 P75: 2.92 P99: 9.61 P99.9: 1481.16 P99.99: 3894.84 ------------------------------------------------------ [ 0, 1 ] 211 0.899% 0.899% ( 1, 2 ] 12224 52.081% 52.980% ########## ( 2, 3 ] 5630 23.987% 76.967% ##### ( 3, 4 ] 2459 10.477% 87.444% ## ( 4, 6 ] 2440 10.396% 97.840% ## ( 6, 10 ] 302 1.287% 99.127% ( 10, 15 ] 86 0.366% 99.493% ( 15, 22 ] 17 0.072% 99.565% ( 76, 110 ] 2 0.009% 99.574% ( 110, 170 ] 7 0.030% 99.604% ( 170, 250 ] 25 0.107% 99.710% ( 250, 380 ] 1 0.004% 99.715% ( 380, 580 ] 4 0.017% 99.732% ( 580, 870 ] 15 0.064% 99.795% ( 870, 1300 ] 20 0.085% 99.881% ( 1300, 1900 ] 15 0.064% 99.945% ( 1900, 2900 ] 8 0.034% 99.979% ( 2900, 4400 ] 4 0.017% 99.996% ( 4400, 6600 ] 1 0.004% 100.000% ** Level 5 read latency histogram (micros): Count: 180 Average: 1466.1056 StdDev: 1511.67 Min: 85 Median: 826.5000 Max: 6993 Percentiles: P50: 826.50 P75: 2344.44 P99: 6930.00 P99.9: 6993.00 P99.99: 6993.00 ------------------------------------------------------ ( 76, 110 ] 6 3.333% 3.333% # ( 110, 170 ] 12 6.667% 10.000% # ( 170, 250 ] 15 8.333% 18.333% ## ( 250, 380 ] 19 10.556% 28.889% ## ( 380, 580 ] 21 11.667% 40.556% ## ( 580, 870 ] 20 11.111% 51.667% ## ( 870, 1300 ] 22 12.222% 63.889% ## ( 1300, 1900 ] 12 6.667% 70.556% # ( 1900, 2900 ] 18 10.000% 80.556% ## ( 2900, 4400 ] 21 11.667% 92.222% ## ( 4400, 6600 ] 12 6.667% 98.889% # ( 6600, 9900 ] 2 1.111% 100.000% ** DB Stats ** Uptime(secs): 40.2 total, 5.4 interval Cumulative writes: 0 writes, 9000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 8.65 GB, 220.56 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:16.394 H:M:S, 40.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 181.82 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.736 H:M:S, 50.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/07/15-06:42:43 ... thread 0: (1000000,10000000) ops and (80893.3,191108.2) ops/second in (12.361958,52.326371) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/13 3.75 GB 81.8 10.1 0.0 10.1 19.7 9.6 0.0 2.1 150.5 292.5 69.07 63.07 172 0.402 10M 16K 0.0 0.0 L1 19/19 2.13 GB 0.0 6.1 5.6 0.5 6.0 5.5 0.2 1.1 223.8 221.0 27.94 26.56 3 9.314 5813K 47K 0.0 0.0 L2 0/0 0.00 KB 0.0 0.4 0.2 0.2 0.4 0.2 3.6 2.0 252.5 252.4 1.50 1.45 2 0.748 383K 197 0.0 0.0 L3 17/0 894.83 MB 0.9 3.5 2.0 1.5 3.5 2.0 1.7 1.7 162.6 162.1 21.80 15.39 15 1.453 3570K 9892 0.0 0.0 L4 163/0 9.73 GB 1.0 2.4 1.9 0.5 2.4 1.9 1.6 1.3 145.5 144.8 16.86 11.34 24 0.703 2433K 10K 0.0 0.0 L5 1084/0 89.14 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1299/32 105.62 GB 0.0 22.5 9.6 12.8 32.0 19.1 10.5 3.3 167.8 238.6 137.17 117.81 216 0.635 22M 83K 0.0 0.0 Int 0/0 0.00 KB 0.0 22.5 9.6 12.8 32.0 19.1 10.3 3.3 167.8 238.6 137.17 117.81 216 0.635 22M 83K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 22.5 9.6 12.8 22.4 9.5 0.0 0.0 213.6 212.5 107.78 93.02 76 1.418 22M 83K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 9.6 9.6 0.0 0.0 0.0 334.4 29.39 24.79 140 0.210 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 52.5 total, 52.3 interval Flush(GB): cumulative 9.597, interval 9.597 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 31.97 GB write, 623.22 MB/s write, 22.48 GB read, 438.31 MB/s read, 137.2 seconds Interval compaction: 31.97 GB write, 625.52 MB/s write, 22.48 GB read, 439.93 MB/s read, 137.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 31, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 31, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 50, pending-compaction-bytes-delays: 7, pending-compaction-bytes-stops: 0, total-delays: 38, total-stops: 50, interval: 88 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 4539155 Average: 2.4648 StdDev: 18.82 Min: 1 Median: 1.4639 Max: 5611 Percentiles: P50: 1.46 P75: 1.85 P99: 6.14 P99.9: 98.03 P99.99: 274.33 ------------------------------------------------------ [ 0, 1 ] 892793 19.669% 19.669% #### ( 1, 2 ] 2968036 65.387% 85.056% ############# ( 2, 3 ] 403478 8.889% 93.945% ## ( 3, 4 ] 144837 3.191% 97.136% # ( 4, 6 ] 83442 1.838% 98.974% ( 6, 10 ] 33932 0.748% 99.722% ( 10, 15 ] 5563 0.123% 99.844% ( 15, 22 ] 1048 0.023% 99.867% ( 22, 34 ] 63 0.001% 99.869% ( 34, 51 ] 52 0.001% 99.870% ( 51, 76 ] 116 0.003% 99.872% ( 76, 110 ] 1943 0.043% 99.915% ( 110, 170 ] 2611 0.058% 99.973% ( 170, 250 ] 692 0.015% 99.988% ( 250, 380 ] 524 0.012% 99.999% ( 380, 580 ] 465 0.010% 100.010% ( 580, 870 ] 261 0.006% 100.015% ( 870, 1300 ] 142 0.003% 100.019% ( 1300, 1900 ] 55 0.001% 100.020% ( 1900, 2900 ] 46 0.001% 100.021% ( 2900, 4400 ] 43 0.001% 100.022% ( 4400, 6600 ] 7 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 1057559 Average: 4.8114 StdDev: 80.44 Min: 1 Median: 1.7017 Max: 10767 Percentiles: P50: 1.70 P75: 2.46 P99: 8.99 P99.9: 107.56 P99.99: 369.22 ------------------------------------------------------ [ 0, 1 ] 104803 9.910% 9.910% ## ( 1, 2 ] 604175 57.129% 67.039% ########### ( 2, 3 ] 181255 17.139% 84.178% ### ( 3, 4 ] 73665 6.966% 91.144% # ( 4, 6 ] 68945 6.519% 97.663% # ( 6, 10 ] 18947 1.792% 99.454% ( 10, 15 ] 3770 0.356% 99.811% ( 15, 22 ] 444 0.042% 99.853% ( 22, 34 ] 22 0.002% 99.855% ( 34, 51 ] 13 0.001% 99.856% ( 51, 76 ] 68 0.006% 99.863% ( 76, 110 ] 426 0.040% 99.903% ( 110, 170 ] 514 0.049% 99.952% ( 170, 250 ] 233 0.022% 99.974% ( 250, 380 ] 190 0.018% 99.992% ( 380, 580 ] 236 0.022% 100.014% ( 580, 870 ] 238 0.023% 100.036% ( 870, 1300 ] 155 0.015% 100.051% ( 1300, 1900 ] 105 0.010% 100.061% ( 1900, 2900 ] 175 0.017% 100.078% ( 2900, 4400 ] 191 0.018% 100.096% ( 4400, 6600 ] 62 0.006% 100.101% ( 6600, 9900 ] 19 0.002% 100.103% ( 9900, 14000 ] 1 0.000% 100.103% ** Level 2 read latency histogram (micros): Count: 95914 Average: 3.4081 StdDev: 63.41 Min: 1 Median: 1.3943 Max: 5920 Percentiles: P50: 1.39 P75: 1.80 P99: 5.65 P99.9: 155.43 P99.99: 3914.11 ------------------------------------------------------ [ 0, 1 ] 24550 25.596% 25.596% ##### ( 1, 2 ] 59364 61.893% 87.489% ############ ( 2, 3 ] 6689 6.974% 94.463% # ( 3, 4 ] 2879 3.002% 97.464% # ( 4, 6 ] 1786 1.862% 99.326% ( 6, 10 ] 434 0.452% 99.779% ( 10, 15 ] 94 0.098% 99.877% ( 15, 22 ] 5 0.005% 99.882% ( 22, 34 ] 3 0.003% 99.885% ( 51, 76 ] 2 0.002% 99.887% ( 76, 110 ] 3 0.003% 99.891% ( 110, 170 ] 12 0.013% 99.903% ( 170, 250 ] 9 0.009% 99.912% ( 250, 380 ] 10 0.010% 99.923% ( 380, 580 ] 15 0.016% 99.938% ( 580, 870 ] 11 0.011% 99.950% ( 870, 1300 ] 16 0.017% 99.967% ( 1300, 1900 ] 8 0.008% 99.975% ( 1900, 2900 ] 9 0.009% 99.984% ( 2900, 4400 ] 8 0.008% 99.993% ( 4400, 6600 ] 7 0.007% 100.000% ** Level 3 read latency histogram (micros): Count: 635382 Average: 10.7679 StdDev: 138.58 Min: 0 Median: 1.8894 Max: 9391 Percentiles: P50: 1.89 P75: 2.87 P99: 10.77 P99.9: 992.89 P99.99: 1599.00 ------------------------------------------------------ [ 0, 1 ] 20391 3.209% 3.209% # ( 1, 2 ] 334254 52.607% 55.816% ########### ( 2, 3 ] 140140 22.056% 77.872% #### ( 3, 4 ] 64323 10.124% 87.996% ## ( 4, 6 ] 57718 9.084% 97.080% ## ( 6, 10 ] 11842 1.864% 98.943% ( 10, 15 ] 2347 0.369% 99.313% ( 15, 22 ] 313 0.049% 99.362% ( 22, 34 ] 86 0.014% 99.375% ( 34, 51 ] 39 0.006% 99.382% ( 51, 76 ] 15 0.002% 99.384% ( 76, 110 ] 164 0.026% 99.410% ( 110, 170 ] 460 0.072% 99.482% ( 170, 250 ] 415 0.065% 99.548% ( 250, 380 ] 543 0.085% 99.633% ( 380, 580 ] 731 0.115% 99.748% ( 580, 870 ] 801 0.126% 99.874% ( 870, 1300 ] 576 0.091% 99.965% ( 1300, 1900 ] 322 0.051% 100.015% ( 1900, 2900 ] 333 0.052% 100.068% ( 2900, 4400 ] 339 0.053% 100.121% ( 4400, 6600 ] 106 0.017% 100.138% ( 6600, 9900 ] 19 0.003% 100.141% ** Level 4 read latency histogram (micros): Count: 45396 Average: 7.2989 StdDev: 81.18 Min: 0 Median: 1.9938 Max: 4479 Percentiles: P50: 1.99 P75: 2.99 P99: 13.17 P99.9: 1096.81 P99.99: 2515.32 ------------------------------------------------------ [ 0, 1 ] 441 0.971% 0.971% ( 1, 2 ] 22395 49.333% 50.304% ########## ( 2, 3 ] 11284 24.857% 75.161% ##### ( 3, 4 ] 4941 10.884% 86.045% ## ( 4, 6 ] 4819 10.615% 96.660% ## ( 6, 10 ] 894 1.969% 98.630% ( 10, 15 ] 265 0.584% 99.214% ( 15, 22 ] 28 0.062% 99.275% ( 22, 34 ] 1 0.002% 99.277% ( 76, 110 ] 29 0.064% 99.341% ( 110, 170 ] 69 0.152% 99.493% ( 170, 250 ] 74 0.163% 99.656% ( 250, 380 ] 35 0.077% 99.733% ( 380, 580 ] 32 0.070% 99.804% ( 580, 870 ] 32 0.070% 99.874% ( 870, 1300 ] 22 0.048% 99.923% ( 1300, 1900 ] 20 0.044% 99.967% ( 1900, 2900 ] 17 0.037% 100.004% ( 2900, 4400 ] 5 0.011% 100.015% ( 4400, 6600 ] 1 0.002% 100.018% ** Level 5 read latency histogram (micros): Count: 200 Average: 1388.0350 StdDev: 1477.00 Min: 85 Median: 751.3636 Max: 6993 Percentiles: P50: 751.36 P75: 2163.16 P99: 6600.00 P99.9: 6993.00 P99.99: 6993.00 ------------------------------------------------------ ( 76, 110 ] 8 4.000% 4.000% # ( 110, 170 ] 16 8.000% 12.000% ## ( 170, 250 ] 18 9.000% 21.000% ## ( 250, 380 ] 21 10.500% 31.500% ## ( 380, 580 ] 24 12.000% 43.500% ## ( 580, 870 ] 22 11.000% 54.500% ## ( 870, 1300 ] 22 11.000% 65.500% ## ( 1300, 1900 ] 14 7.000% 72.500% # ( 1900, 2900 ] 19 9.500% 82.000% ## ( 2900, 4400 ] 22 11.000% 93.000% ## ( 4400, 6600 ] 12 6.000% 99.000% # ( 6600, 9900 ] 2 1.000% 100.000% ** DB Stats ** Uptime(secs): 52.5 total, 12.4 interval Cumulative writes: 0 writes, 10M keys, 0 commit groups, 0.0 writes per commit group, ingest: 9.61 GB, 187.38 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:25.597 H:M:S, 48.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 79.61 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:9.202 H:M:S, 74.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] overwrite : 5.975 micros/op 167359 ops/sec 60.044 seconds 10048999 operations; 162.2 MB/s Microseconds per write: Count: 10048999 Average: 5.9752 StdDev: 387.78 Min: 0 Median: 1.7182 Max: 680504 Percentiles: P50: 1.72 P75: 2.67 P99: 9.25 P99.9: 1170.29 P99.99: 2700.69 ------------------------------------------------------ [ 0, 1 ] 1447765 14.407% 14.407% ### ( 1, 2 ] 4979950 49.557% 63.964% ########## ( 2, 3 ] 1649017 16.410% 80.373% ### ( 3, 4 ] 1323013 13.166% 93.539% ### ( 4, 6 ] 453613 4.514% 98.053% # ( 6, 10 ] 117254 1.167% 99.220% ( 10, 15 ] 35938 0.358% 99.578% ( 15, 22 ] 7985 0.079% 99.657% ( 22, 34 ] 1858 0.018% 99.676% ( 34, 51 ] 261 0.003% 99.678% ( 51, 76 ] 40 0.000% 99.679% ( 76, 110 ] 7392 0.074% 99.752% ( 110, 170 ] 2399 0.024% 99.776% ( 170, 250 ] 272 0.003% 99.779% ( 250, 380 ] 150 0.001% 99.780% ( 380, 580 ] 16 0.000% 99.780% ( 580, 870 ] 19 0.000% 99.781% ( 870, 1300 ] 17195 0.171% 99.952% ( 1300, 1900 ] 29 0.000% 99.952% ( 1900, 2900 ] 4781 0.048% 99.999% ( 2900, 4400 ] 21 0.000% 100.000% ( 4400, 6600 ] 2 0.000% 100.000% ( 6600, 9900 ] 3 0.000% 100.000% ( 9900, 14000 ] 1 0.000% 100.000% ( 14000, 22000 ] 6 0.000% 100.000% ( 22000, 33000 ] 1 0.000% 100.000% ( 110000, 170000 ] 2 0.000% 100.000% ( 170000, 250000 ] 14 0.000% 100.000% ( 570000, 860000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 0 rocksdb.bloom.filter.full.positive COUNT : 0 rocksdb.bloom.filter.full.true.positive COUNT : 0 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 0 rocksdb.memtable.miss COUNT : 0 rocksdb.l0.hit COUNT : 0 rocksdb.l1.hit COUNT : 0 rocksdb.l2andup.hit COUNT : 0 rocksdb.compaction.key.drop.new COUNT : 188383 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 10048999 rocksdb.number.keys.read COUNT : 0 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 10370566968 rocksdb.bytes.read COUNT : 0 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 475 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 32860076 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 28045462342 rocksdb.compact.write.bytes COUNT : 27869249182 rocksdb.flush.write.bytes COUNT : 10371205204 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 51 rocksdb.number.superversion_acquires COUNT : 0 rocksdb.number.superversion_releases COUNT : 0 rocksdb.number.superversion_cleanups COUNT : 0 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 261 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 10287438848 rocksdb.memtable.garbage.bytes.at.flush COUNT : 3861504 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 28137336040 rocksdb.non.last.level.read.count COUNT : 6798298 rocksdb.block.checksum.compute.count COUNT : 6797399 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.micros P50 : 1.356539 P95 : 3.845476 P99 : 7.732647 P100 : 680500.000000 COUNT : 10048999 SUM : 55412873 rocksdb.compaction.times.micros P50 : 1054285.714286 P95 : 3926666.666667 P99 : 14365373.000000 P100 : 14365373.000000 COUNT : 76 SUM : 107779269 rocksdb.compaction.times.cpu_micros P50 : 928000.000000 P95 : 3670000.000000 P99 : 12404000.000000 P100 : 13475127.000000 COUNT : 76 SUM : 93017717 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 687.470588 P95 : 1258.954545 P99 : 16360.000000 P100 : 21614.000000 COUNT : 141 SUM : 152810 rocksdb.compaction.outfile.sync.micros P50 : 864.589552 P95 : 29150.000000 P99 : 69250.000000 P100 : 193456.000000 COUNT : 315 SUM : 1661291 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 103.602151 P95 : 698.416667 P99 : 3145.000000 P100 : 4023.000000 COUNT : 251 SUM : 58391 rocksdb.table.open.io.micros P50 : 1338.571429 P95 : 8009.375000 P99 : 12975.000000 P100 : 19453.000000 COUNT : 475 SUM : 1007093 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.554303 P95 : 6.901968 P99 : 10.839658 P100 : 10828.000000 COUNT : 6795955 SUM : 39267904 rocksdb.read.block.get.micros P50 : 380.000000 P95 : 3444.117647 P99 : 5330.769231 P100 : 7072.000000 COUNT : 950 SUM : 796956 rocksdb.write.raw.block.micros P50 : 0.512448 P95 : 0.973651 P99 : 2.812790 P100 : 18863.000000 COUNT : 9261385 SUM : 18038717 rocksdb.numfiles.in.singlecompaction P50 : 3.253521 P95 : 5.480000 P99 : 14.566667 P100 : 32.000000 COUNT : 126 SUM : 472 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.560031 P95 : 1162.245232 P99 : 2472.929441 P100 : 680315.000000 COUNT : 206171 SUM : 32879164 rocksdb.sst.read.micros P50 : 1.520660 P95 : 3.959445 P99 : 8.128633 P100 : 10767.000000 COUNT : 6798343 SUM : 25594438 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 10048999 SUM : 10370566968 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 207021.276596 P95 : 355218.750000 P99 : 655550.000000 P100 : 850409.000000 COUNT : 141 SUM : 29568672 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 297 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0