[2024-11-30 15:49:21.240398] Starting SPDK v25.01-pre git sha1 35cd3e84d / DPDK 24.11.0-rc4 initialization... [2024-11-30 15:49:21.240664] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1394355 ] Set seed to 1732978161233745 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-30 15:49:21.412445] pci_dpdk.c: 37:dpdk_pci_init: *NOTICE*: In-development DPDK 24.11.0-rc4 is used. There is no support for it in SPDK. Enabled only for validation. [2024-11-30 15:49:21.461984] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-30 15:49:21.485329] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-30 15:49:21.485372] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1394355' to capture a snapshot of events at runtime. [2024-11-30 15:49:21.485384] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-30 15:49:21.485393] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-30 15:49:21.485403] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1394355 for offline analysis/debug. [2024-11-30 15:49:21.485941] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Nov 30 15:49:25 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/30-15:49:33 ... thread 0: (1000000,1000000) ops and (123637.9,123637.9) ops/second in (8.088134,8.088134) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/10 1.74 GB 50.3 0.4 0.0 0.4 1.2 0.8 0.0 1.5 37.8 119.6 10.00 3.92 10 1.000 383K 524 0.0 0.0 L1 4/4 454.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 2/1 257.56 MB 1.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 36/9 2.11 GB 1.6 2.8 1.7 1.1 2.8 1.7 0.0 1.6 96.4 95.7 29.92 13.32 14 2.137 1265K 0 0.0 0.0 L4 181/7 12.01 GB 1.2 3.3 2.4 0.9 3.3 2.4 0.1 1.4 106.8 106.2 32.09 15.81 24 1.337 1470K 7 0.0 0.0 L5 1080/0 89.26 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1315/31 105.82 GB 0.0 6.5 4.1 2.4 7.3 4.9 0.1 9.1 92.9 103.7 72.01 33.06 48 1.500 3119K 531 0.0 0.0 Int 0/0 0.00 KB 0.0 6.5 4.1 2.4 7.3 4.9 0.1 9.1 92.9 103.7 72.01 33.06 48 1.500 3119K 531 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.5 4.1 2.4 6.5 4.1 0.0 0.0 103.3 102.7 64.77 30.63 39 1.661 3119K 531 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 113.0 7.25 2.43 9 0.805 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.2 total, 8.1 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.29 GB write, 906.89 MB/s write, 6.53 GB read, 812.65 MB/s read, 72.0 seconds Interval compaction: 7.29 GB write, 923.00 MB/s write, 6.53 GB read, 827.08 MB/s read, 72.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 32, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 32, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 342981 Average: 12.1666 StdDev: 162.15 Min: 1 Median: 1.9049 Max: 35768 Percentiles: P50: 1.90 P75: 2.91 P99: 11.13 P99.9: 1844.18 P99.99: 3606.38 ------------------------------------------------------ [ 0, 1 ] 12096 3.527% 3.527% # ( 1, 2 ] 176141 51.356% 54.883% ########## ( 2, 3 ] 75877 22.123% 77.005% #### ( 3, 4 ] 34255 9.987% 86.993% ## ( 4, 6 ] 33378 9.732% 96.725% ## ( 6, 10 ] 7554 2.202% 98.927% ( 10, 15 ] 1114 0.325% 99.252% ( 15, 22 ] 129 0.038% 99.289% ( 22, 34 ] 11 0.003% 99.293% ( 34, 51 ] 11 0.003% 99.296% ( 51, 76 ] 14 0.004% 99.300% ( 76, 110 ] 9 0.003% 99.303% ( 110, 170 ] 32 0.009% 99.312% ( 170, 250 ] 96 0.028% 99.340% ( 250, 380 ] 192 0.056% 99.396% ( 380, 580 ] 381 0.111% 99.507% ( 580, 870 ] 530 0.155% 99.661% ( 870, 1300 ] 517 0.151% 99.812% ( 1300, 1900 ] 333 0.097% 99.909% ( 1900, 2900 ] 200 0.058% 99.968% ( 2900, 4400 ] 165 0.048% 100.016% ( 4400, 6600 ] 53 0.015% 100.031% ( 6600, 9900 ] 6 0.002% 100.033% ( 9900, 14000 ] 1 0.000% 100.033% ( 14000, 22000 ] 1 0.000% 100.034% ( 22000, 33000 ] 1 0.000% 100.034% ( 33000, 50000 ] 1 0.000% 100.034% ** Level 1 read latency histogram (micros): Count: 51344 Average: 14.1510 StdDev: 269.52 Min: 1 Median: 1.9564 Max: 50375 Percentiles: P50: 1.96 P75: 2.95 P99: 13.16 P99.9: 2501.10 P99.99: 6173.12 ------------------------------------------------------ [ 0, 1 ] 1078 2.100% 2.100% ( 1, 2 ] 25716 50.086% 52.185% ########## ( 2, 3 ] 12339 24.032% 76.217% ##### ( 3, 4 ] 5155 10.040% 86.257% ## ( 4, 6 ] 5073 9.880% 96.138% ## ( 6, 10 ] 1337 2.604% 98.742% # ( 10, 15 ] 210 0.409% 99.151% ( 15, 22 ] 30 0.058% 99.209% ( 22, 34 ] 5 0.010% 99.219% ( 34, 51 ] 3 0.006% 99.225% ( 51, 76 ] 4 0.008% 99.233% ( 76, 110 ] 2 0.004% 99.237% ( 110, 170 ] 4 0.008% 99.244% ( 170, 250 ] 6 0.012% 99.256% ( 250, 380 ] 20 0.039% 99.295% ( 380, 580 ] 54 0.105% 99.400% ( 580, 870 ] 88 0.171% 99.572% ( 870, 1300 ] 85 0.166% 99.737% ( 1300, 1900 ] 53 0.103% 99.840% ( 1900, 2900 ] 51 0.099% 99.940% ( 2900, 4400 ] 17 0.033% 99.973% ( 4400, 6600 ] 11 0.021% 99.994% ( 6600, 9900 ] 2 0.004% 99.998% ( 50000, 75000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 346094 Average: 11.1060 StdDev: 141.86 Min: 1 Median: 1.9618 Max: 12423 Percentiles: P50: 1.96 P75: 2.98 P99: 11.82 P99.9: 1451.42 P99.99: 2414.24 ------------------------------------------------------ [ 0, 1 ] 11024 3.185% 3.185% # ( 1, 2 ] 168461 48.675% 51.860% ########## ( 2, 3 ] 81731 23.615% 75.475% ##### ( 3, 4 ] 35361 10.217% 85.693% ## ( 4, 6 ] 35046 10.126% 95.819% ## ( 6, 10 ] 10332 2.985% 98.804% # ( 10, 15 ] 1862 0.538% 99.342% ( 15, 22 ] 303 0.088% 99.430% ( 22, 34 ] 78 0.023% 99.452% ( 34, 51 ] 33 0.010% 99.462% ( 51, 76 ] 21 0.006% 99.468% ( 76, 110 ] 27 0.008% 99.476% ( 110, 170 ] 87 0.025% 99.501% ( 170, 250 ] 93 0.027% 99.528% ( 250, 380 ] 185 0.053% 99.581% ( 380, 580 ] 290 0.084% 99.665% ( 580, 870 ] 380 0.110% 99.775% ( 870, 1300 ] 364 0.105% 99.880% ( 1300, 1900 ] 277 0.080% 99.960% ( 1900, 2900 ] 203 0.059% 100.018% ( 2900, 4400 ] 175 0.051% 100.069% ( 4400, 6600 ] 62 0.018% 100.087% ( 6600, 9900 ] 8 0.002% 100.089% ( 9900, 14000 ] 1 0.000% 100.090% ** Level 3 read latency histogram (micros): Count: 930563 Average: 12.7725 StdDev: 166.88 Min: 1 Median: 1.9911 Max: 42082 Percentiles: P50: 1.99 P75: 2.99 P99: 9.86 P99.9: 1151.59 P99.99: 1582.80 ------------------------------------------------------ [ 0, 1 ] 16698 1.794% 1.794% ( 1, 2 ] 452626 48.640% 50.434% ########## ( 2, 3 ] 231326 24.859% 75.293% ##### ( 3, 4 ] 101226 10.878% 86.171% ## ( 4, 6 ] 97362 10.463% 96.634% ## ( 6, 10 ] 22819 2.452% 99.086% ( 10, 15 ] 3694 0.397% 99.483% ( 15, 22 ] 466 0.050% 99.533% ( 22, 34 ] 54 0.006% 99.539% ( 34, 51 ] 44 0.005% 99.544% ( 51, 76 ] 52 0.006% 99.549% ( 76, 110 ] 24 0.003% 99.552% ( 110, 170 ] 78 0.008% 99.560% ( 170, 250 ] 173 0.019% 99.579% ( 250, 380 ] 378 0.041% 99.619% ( 380, 580 ] 744 0.080% 99.699% ( 580, 870 ] 1064 0.114% 99.814% ( 870, 1300 ] 1233 0.133% 99.946% ( 1300, 1900 ] 874 0.094% 100.040% ( 1900, 2900 ] 717 0.077% 100.117% ( 2900, 4400 ] 624 0.067% 100.184% ( 4400, 6600 ] 161 0.017% 100.201% ( 6600, 9900 ] 41 0.004% 100.206% ( 9900, 14000 ] 7 0.001% 100.207% ( 14000, 22000 ] 1 0.000% 100.207% ( 33000, 50000 ] 1 0.000% 100.207% ** Level 4 read latency histogram (micros): Count: 284991 Average: 26.2903 StdDev: 318.27 Min: 0 Median: 1.9771 Max: 59938 Percentiles: P50: 1.98 P75: 3.00 P99: 690.89 P99.9: 3069.57 P99.99: 4014.87 ------------------------------------------------------ [ 0, 1 ] 5204 1.826% 1.826% ( 1, 2 ] 140505 49.302% 51.128% ########## ( 2, 3 ] 68033 23.872% 75.000% ##### ( 3, 4 ] 30670 10.762% 85.761% ## ( 4, 6 ] 28549 10.018% 95.779% ## ( 6, 10 ] 6569 2.305% 98.084% ( 10, 15 ] 944 0.331% 98.415% ( 15, 22 ] 135 0.047% 98.462% ( 22, 34 ] 10 0.004% 98.466% ( 34, 51 ] 5 0.002% 98.468% ( 51, 76 ] 10 0.004% 98.471% ( 76, 110 ] 13 0.005% 98.476% ( 110, 170 ] 66 0.023% 98.499% ( 170, 250 ] 172 0.060% 98.559% ( 250, 380 ] 314 0.110% 98.669% ( 380, 580 ] 601 0.211% 98.880% ( 580, 870 ] 892 0.313% 99.193% ( 870, 1300 ] 907 0.318% 99.512% ( 1300, 1900 ] 598 0.210% 99.721% ( 1900, 2900 ] 463 0.162% 99.884% ( 2900, 4400 ] 407 0.143% 100.027% ( 4400, 6600 ] 127 0.045% 100.071% ( 6600, 9900 ] 24 0.008% 100.080% ( 9900, 14000 ] 1 0.000% 100.080% ( 14000, 22000 ] 3 0.001% 100.081% ( 22000, 33000 ] 1 0.000% 100.081% ( 33000, 50000 ] 4 0.001% 100.083% ( 50000, 75000 ] 1 0.000% 100.083% ** Level 5 read latency histogram (micros): Count: 14571 Average: 1585.2779 StdDev: 1531.45 Min: 91 Median: 1094.1571 Max: 56951 Percentiles: P50: 1094.16 P75: 2119.52 P99: 6464.34 P99.9: 9742.87 P99.99: 56786.25 ------------------------------------------------------ ( 76, 110 ] 12 0.082% 0.082% ( 110, 170 ] 134 0.920% 1.002% ( 170, 250 ] 312 2.141% 3.143% ( 250, 380 ] 836 5.737% 8.881% # ( 380, 580 ] 1855 12.731% 21.611% ### ( 580, 870 ] 2680 18.393% 40.004% #### ( 870, 1300 ] 2794 19.175% 59.179% #### ( 1300, 1900 ] 1947 13.362% 72.541% ### ( 1900, 2900 ] 1632 11.200% 83.742% ## ( 2900, 4400 ] 1619 11.111% 94.853% ## ( 4400, 6600 ] 644 4.420% 99.273% # ( 6600, 9900 ] 96 0.659% 99.931% ( 9900, 14000 ] 5 0.034% 99.966% ( 14000, 22000 ] 3 0.021% 99.986% ( 50000, 75000 ] 2 0.014% 100.000% ** DB Stats ** Uptime(secs): 8.2 total, 8.1 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 119.52 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.235 H:M:S, 51.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 121.64 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.235 H:M:S, 52.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/30-15:49:38 ... thread 0: (1000000,2000000) ops and (218378.9,157886.5) ops/second in (4.579197,12.667331) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 2.78 GB 138.4 1.4 0.0 1.4 3.2 1.8 0.0 1.7 69.2 163.4 20.04 10.91 28 0.716 1407K 1809 0.0 0.0 L1 4/4 454.19 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 890.68 MB 0.9 3.3 2.0 1.3 3.2 1.9 0.0 1.6 100.6 99.8 33.18 15.45 16 2.074 1724K 3246 0.0 0.0 L4 159/0 9.45 GB 1.0 6.0 3.7 2.3 5.9 3.6 0.3 1.6 118.0 116.6 52.12 28.08 43 1.212 2556K 7 0.0 0.0 L5 1128/0 93.26 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1325/18 106.81 GB 0.0 10.6 5.6 5.0 12.4 7.4 4.3 6.7 103.2 120.2 105.34 54.44 87 1.211 5687K 5062 0.0 0.0 Int 0/0 0.00 KB 0.0 10.6 5.6 5.0 12.4 7.4 4.3 6.7 103.2 120.2 105.34 54.44 87 1.211 5687K 5062 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.6 5.6 5.0 10.5 5.5 0.0 0.0 116.4 115.3 93.44 49.07 63 1.483 5687K 5062 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 158.8 11.90 5.37 24 0.496 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.8 total, 12.7 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.37 GB write, 988.24 MB/s write, 10.62 GB read, 848.57 MB/s read, 105.3 seconds Interval compaction: 12.37 GB write, 999.45 MB/s write, 10.62 GB read, 858.19 MB/s read, 105.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 42, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 661242 Average: 13.9081 StdDev: 145.98 Min: 0 Median: 1.7922 Max: 35768 Percentiles: P50: 1.79 P75: 2.71 P99: 256.18 P99.9: 1493.64 P99.99: 2600.52 ------------------------------------------------------ [ 0, 1 ] 46604 7.048% 7.048% # ( 1, 2 ] 358531 54.221% 61.269% ########### ( 2, 3 ] 128132 19.377% 80.646% #### ( 3, 4 ] 54348 8.219% 88.865% ## ( 4, 6 ] 48863 7.390% 96.255% # ( 6, 10 ] 11101 1.679% 97.934% ( 10, 15 ] 1739 0.263% 98.197% ( 15, 22 ] 260 0.039% 98.236% ( 22, 34 ] 24 0.004% 98.240% ( 34, 51 ] 28 0.004% 98.244% ( 51, 76 ] 25 0.004% 98.248% ( 76, 110 ] 542 0.082% 98.330% ( 110, 170 ] 2496 0.377% 98.707% ( 170, 250 ] 1864 0.282% 98.989% ( 250, 380 ] 1548 0.234% 99.223% ( 380, 580 ] 1736 0.263% 99.486% ( 580, 870 ] 1556 0.235% 99.721% ( 870, 1300 ] 1015 0.153% 99.874% ( 1300, 1900 ] 526 0.080% 99.954% ( 1900, 2900 ] 341 0.052% 100.006% ( 2900, 4400 ] 294 0.044% 100.050% ( 4400, 6600 ] 79 0.012% 100.062% ( 6600, 9900 ] 7 0.001% 100.063% ( 9900, 14000 ] 1 0.000% 100.063% ( 14000, 22000 ] 1 0.000% 100.063% ( 22000, 33000 ] 1 0.000% 100.064% ( 33000, 50000 ] 1 0.000% 100.064% ** Level 1 read latency histogram (micros): Count: 66915 Average: 13.2518 StdDev: 240.36 Min: 1 Median: 2.0316 Max: 50375 Percentiles: P50: 2.03 P75: 3.04 P99: 21.85 P99.9: 2297.83 P99.99: 5923.22 ------------------------------------------------------ [ 0, 1 ] 1581 2.363% 2.363% ( 1, 2 ] 31340 46.836% 49.198% ######### ( 2, 3 ] 16975 25.368% 74.566% ##### ( 3, 4 ] 7508 11.220% 85.786% ## ( 4, 6 ] 6613 9.883% 95.669% ## ( 6, 10 ] 1854 2.771% 98.440% # ( 10, 15 ] 321 0.480% 98.920% ( 15, 22 ] 55 0.082% 99.002% ( 22, 34 ] 9 0.013% 99.015% ( 34, 51 ] 3 0.004% 99.020% ( 51, 76 ] 4 0.006% 99.026% ( 76, 110 ] 20 0.030% 99.056% ( 110, 170 ] 69 0.103% 99.159% ( 170, 250 ] 52 0.078% 99.236% ( 250, 380 ] 61 0.091% 99.328% ( 380, 580 ] 98 0.146% 99.474% ( 580, 870 ] 106 0.158% 99.632% ( 870, 1300 ] 99 0.148% 99.780% ( 1300, 1900 ] 59 0.088% 99.868% ( 1900, 2900 ] 53 0.079% 99.948% ( 2900, 4400 ] 20 0.030% 99.978% ( 4400, 6600 ] 12 0.018% 99.996% ( 6600, 9900 ] 2 0.003% 99.999% ( 50000, 75000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 378812 Average: 10.4908 StdDev: 135.91 Min: 1 Median: 1.9570 Max: 12423 Percentiles: P50: 1.96 P75: 2.97 P99: 11.39 P99.9: 1384.58 P99.99: 2403.05 ------------------------------------------------------ [ 0, 1 ] 11657 3.077% 3.077% # ( 1, 2 ] 185730 49.030% 52.107% ########## ( 2, 3 ] 89808 23.708% 75.815% ##### ( 3, 4 ] 38234 10.093% 85.908% ## ( 4, 6 ] 37842 9.990% 95.897% ## ( 6, 10 ] 11207 2.958% 98.856% # ( 10, 15 ] 1964 0.518% 99.374% ( 15, 22 ] 310 0.082% 99.456% ( 22, 34 ] 81 0.021% 99.478% ( 34, 51 ] 33 0.009% 99.486% ( 51, 76 ] 21 0.006% 99.492% ( 76, 110 ] 30 0.008% 99.500% ( 110, 170 ] 100 0.026% 99.526% ( 170, 250 ] 105 0.028% 99.554% ( 250, 380 ] 203 0.054% 99.607% ( 380, 580 ] 313 0.083% 99.690% ( 580, 870 ] 386 0.102% 99.792% ( 870, 1300 ] 370 0.098% 99.890% ( 1300, 1900 ] 278 0.073% 99.963% ( 1900, 2900 ] 203 0.054% 100.017% ( 2900, 4400 ] 176 0.046% 100.063% ( 4400, 6600 ] 62 0.016% 100.079% ( 6600, 9900 ] 8 0.002% 100.082% ( 9900, 14000 ] 1 0.000% 100.082% ** Level 3 read latency histogram (micros): Count: 1227405 Average: 11.2639 StdDev: 150.36 Min: 1 Median: 1.9656 Max: 42082 Percentiles: P50: 1.97 P75: 2.95 P99: 9.75 P99.9: 1023.43 P99.99: 1415.89 ------------------------------------------------------ [ 0, 1 ] 23761 1.936% 1.936% ( 1, 2 ] 610970 49.777% 51.713% ########## ( 2, 3 ] 299742 24.421% 76.134% ##### ( 3, 4 ] 128967 10.507% 86.641% ## ( 4, 6 ] 123871 10.092% 96.733% ## ( 6, 10 ] 29657 2.416% 99.150% ( 10, 15 ] 4466 0.364% 99.514% ( 15, 22 ] 554 0.045% 99.559% ( 22, 34 ] 68 0.006% 99.564% ( 34, 51 ] 58 0.005% 99.569% ( 51, 76 ] 82 0.007% 99.576% ( 76, 110 ] 67 0.005% 99.581% ( 110, 170 ] 234 0.019% 99.600% ( 170, 250 ] 306 0.025% 99.625% ( 250, 380 ] 537 0.044% 99.669% ( 380, 580 ] 983 0.080% 99.749% ( 580, 870 ] 1344 0.109% 99.858% ( 870, 1300 ] 1431 0.117% 99.975% ( 1300, 1900 ] 954 0.078% 100.053% ( 1900, 2900 ] 769 0.063% 100.115% ( 2900, 4400 ] 677 0.055% 100.171% ( 4400, 6600 ] 170 0.014% 100.184% ( 6600, 9900 ] 41 0.003% 100.188% ( 9900, 14000 ] 7 0.001% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 33000, 50000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 612932 Average: 18.8596 StdDev: 233.39 Min: 0 Median: 1.9413 Max: 59938 Percentiles: P50: 1.94 P75: 2.95 P99: 344.03 P99.9: 1864.31 P99.99: 2773.20 ------------------------------------------------------ [ 0, 1 ] 12668 2.067% 2.067% ( 1, 2 ] 312113 50.921% 52.988% ########## ( 2, 3 ] 142536 23.255% 76.243% ##### ( 3, 4 ] 61586 10.048% 86.291% ## ( 4, 6 ] 57751 9.422% 95.713% ## ( 6, 10 ] 14711 2.400% 98.113% ( 10, 15 ] 1863 0.304% 98.417% ( 15, 22 ] 243 0.040% 98.456% ( 22, 34 ] 31 0.005% 98.461% ( 34, 51 ] 28 0.005% 98.466% ( 51, 76 ] 48 0.008% 98.474% ( 76, 110 ] 336 0.055% 98.529% ( 110, 170 ] 1195 0.195% 98.724% ( 170, 250 ] 1008 0.164% 98.888% ( 250, 380 ] 948 0.155% 99.043% ( 380, 580 ] 1444 0.236% 99.278% ( 580, 870 ] 1645 0.268% 99.547% ( 870, 1300 ] 1423 0.232% 99.779% ( 1300, 1900 ] 789 0.129% 99.908% ( 1900, 2900 ] 578 0.094% 100.002% ( 2900, 4400 ] 526 0.086% 100.088% ( 4400, 6600 ] 150 0.024% 100.112% ( 6600, 9900 ] 27 0.004% 100.117% ( 9900, 14000 ] 1 0.000% 100.117% ( 14000, 22000 ] 3 0.000% 100.117% ( 22000, 33000 ] 1 0.000% 100.117% ( 33000, 50000 ] 4 0.001% 100.118% ( 50000, 75000 ] 1 0.000% 100.118% ** Level 5 read latency histogram (micros): Count: 44491 Average: 804.4303 StdDev: 1118.61 Min: 73 Median: 427.3640 Max: 56951 Percentiles: P50: 427.36 P75: 930.08 P99: 5559.15 P99.9: 8772.24 P99.99: 18135.73 ------------------------------------------------------ ( 51, 76 ] 2 0.004% 0.004% ( 76, 110 ] 2260 5.080% 5.084% # ( 110, 170 ] 8269 18.586% 23.670% #### ( 170, 250 ] 5548 12.470% 36.140% ## ( 250, 380 ] 4757 10.692% 46.832% ## ( 380, 580 ] 5956 13.387% 60.219% ### ( 580, 870 ] 5944 13.360% 73.579% ### ( 870, 1300 ] 4532 10.186% 83.765% ## ( 1300, 1900 ] 2585 5.810% 89.575% # ( 1900, 2900 ] 1943 4.367% 93.943% # ( 2900, 4400 ] 1877 4.219% 98.161% # ( 4400, 6600 ] 710 1.596% 99.757% ( 6600, 9900 ] 98 0.220% 99.978% ( 9900, 14000 ] 5 0.011% 99.989% ( 14000, 22000 ] 3 0.007% 99.996% ( 50000, 75000 ] 2 0.004% 100.000% ** DB Stats ** Uptime(secs): 12.8 total, 4.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 153.62 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.171 H:M:S, 40.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 214.93 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.936 H:M:S, 20.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/30-15:49:42 ... thread 0: (1000000,3000000) ops and (267638.6,182885.4) ops/second in (3.736382,16.403713) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/15 2.77 GB 0.0 2.3 0.0 2.3 5.1 2.8 0.0 1.8 89.2 194.7 26.84 17.22 47 0.571 2430K 2751 0.0 0.0 L1 19/19 1.37 GB 0.0 1.4 0.9 0.4 1.4 0.9 0.0 1.5 91.8 91.3 15.41 7.39 1 15.413 1437K 8101 0.0 0.0 L3 16/0 890.68 MB 0.9 3.3 2.0 1.3 3.2 1.9 0.0 1.6 100.6 99.8 33.18 15.45 16 2.074 1724K 3246 0.0 0.0 L4 159/0 9.45 GB 1.0 6.0 3.7 2.3 5.9 3.6 0.3 1.6 118.0 116.6 52.12 28.08 43 1.212 2556K 7 0.0 0.0 L5 1128/0 93.26 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1337/34 107.72 GB 0.0 13.0 6.6 6.4 15.6 9.2 4.3 5.7 104.2 125.6 127.55 68.14 107 1.192 8148K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.0 6.6 6.4 15.6 9.2 4.3 5.7 104.2 125.6 127.55 68.14 107 1.192 8148K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.0 6.6 6.4 12.9 6.5 0.0 0.0 117.9 117.0 112.73 60.25 68 1.658 8148K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 191.2 14.82 7.89 39 0.380 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.5 total, 16.4 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.65 GB write, 968.13 MB/s write, 12.98 GB read, 803.44 MB/s read, 127.6 seconds Interval compaction: 15.65 GB write, 976.60 MB/s write, 12.98 GB read, 810.48 MB/s read, 127.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 42, interval: 44 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 897437 Average: 13.8711 StdDev: 129.38 Min: 0 Median: 1.7061 Max: 35768 Percentiles: P50: 1.71 P75: 2.53 P99: 268.92 P99.9: 1238.80 P99.99: 2260.73 ------------------------------------------------------ [ 0, 1 ] 102528 11.425% 11.425% ## ( 1, 2 ] 490292 54.632% 66.057% ########### ( 2, 3 ] 150370 16.755% 82.812% ### ( 3, 4 ] 61904 6.898% 89.710% # ( 4, 6 ] 53622 5.975% 95.685% # ( 6, 10 ] 11921 1.328% 97.014% ( 10, 15 ] 1906 0.212% 97.226% ( 15, 22 ] 282 0.031% 97.258% ( 22, 34 ] 32 0.004% 97.261% ( 34, 51 ] 29 0.003% 97.264% ( 51, 76 ] 37 0.004% 97.268% ( 76, 110 ] 2176 0.242% 97.511% ( 110, 170 ] 8192 0.913% 98.424% ( 170, 250 ] 4759 0.530% 98.954% ( 250, 380 ] 2842 0.317% 99.271% ( 380, 580 ] 2768 0.308% 99.579% ( 580, 870 ] 1962 0.219% 99.798% ( 870, 1300 ] 1071 0.119% 99.917% ( 1300, 1900 ] 529 0.059% 99.976% ( 1900, 2900 ] 350 0.039% 100.015% ( 2900, 4400 ] 297 0.033% 100.048% ( 4400, 6600 ] 80 0.009% 100.057% ( 6600, 9900 ] 7 0.001% 100.058% ( 9900, 14000 ] 1 0.000% 100.058% ( 14000, 22000 ] 1 0.000% 100.058% ( 22000, 33000 ] 1 0.000% 100.058% ( 33000, 50000 ] 1 0.000% 100.058% ** Level 1 read latency histogram (micros): Count: 122461 Average: 10.2260 StdDev: 179.13 Min: 1 Median: 2.3865 Max: 50375 Percentiles: P50: 2.39 P75: 3.61 P99: 111.12 P99.9: 1561.04 P99.99: 4904.88 ------------------------------------------------------ [ 0, 1 ] 2762 2.255% 2.255% ( 1, 2 ] 45426 37.094% 39.350% ####### ( 2, 3 ] 33748 27.558% 66.908% ###### ( 3, 4 ] 16134 13.175% 80.083% ### ( 4, 6 ] 17352 14.169% 94.252% ### ( 6, 10 ] 4553 3.718% 97.970% # ( 10, 15 ] 798 0.652% 98.622% ( 15, 22 ] 290 0.237% 98.858% ( 22, 34 ] 45 0.037% 98.895% ( 34, 51 ] 5 0.004% 98.899% ( 51, 76 ] 6 0.005% 98.904% ( 76, 110 ] 111 0.091% 98.995% ( 110, 170 ] 342 0.279% 99.274% ( 170, 250 ] 200 0.163% 99.437% ( 250, 380 ] 139 0.114% 99.551% ( 380, 580 ] 153 0.125% 99.676% ( 580, 870 ] 138 0.113% 99.789% ( 870, 1300 ] 110 0.090% 99.878% ( 1300, 1900 ] 61 0.050% 99.928% ( 1900, 2900 ] 53 0.043% 99.971% ( 2900, 4400 ] 20 0.016% 99.988% ( 4400, 6600 ] 12 0.010% 99.998% ( 6600, 9900 ] 2 0.002% 99.999% ( 50000, 75000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 378812 Average: 10.4908 StdDev: 135.91 Min: 1 Median: 1.9570 Max: 12423 Percentiles: P50: 1.96 P75: 2.97 P99: 11.39 P99.9: 1384.58 P99.99: 2403.05 ------------------------------------------------------ [ 0, 1 ] 11657 3.077% 3.077% # ( 1, 2 ] 185730 49.030% 52.107% ########## ( 2, 3 ] 89808 23.708% 75.815% ##### ( 3, 4 ] 38234 10.093% 85.908% ## ( 4, 6 ] 37842 9.990% 95.897% ## ( 6, 10 ] 11207 2.958% 98.856% # ( 10, 15 ] 1964 0.518% 99.374% ( 15, 22 ] 310 0.082% 99.456% ( 22, 34 ] 81 0.021% 99.478% ( 34, 51 ] 33 0.009% 99.486% ( 51, 76 ] 21 0.006% 99.492% ( 76, 110 ] 30 0.008% 99.500% ( 110, 170 ] 100 0.026% 99.526% ( 170, 250 ] 105 0.028% 99.554% ( 250, 380 ] 203 0.054% 99.607% ( 380, 580 ] 313 0.083% 99.690% ( 580, 870 ] 386 0.102% 99.792% ( 870, 1300 ] 370 0.098% 99.890% ( 1300, 1900 ] 278 0.073% 99.963% ( 1900, 2900 ] 203 0.054% 100.017% ( 2900, 4400 ] 176 0.046% 100.063% ( 4400, 6600 ] 62 0.016% 100.079% ( 6600, 9900 ] 8 0.002% 100.082% ( 9900, 14000 ] 1 0.000% 100.082% ** Level 3 read latency histogram (micros): Count: 1228183 Average: 11.3690 StdDev: 150.41 Min: 1 Median: 1.9662 Max: 42082 Percentiles: P50: 1.97 P75: 2.96 P99: 9.84 P99.9: 1023.58 P99.99: 1415.84 ------------------------------------------------------ [ 0, 1 ] 23761 1.935% 1.935% ( 1, 2 ] 610986 49.747% 51.682% ########## ( 2, 3 ] 299788 24.409% 76.091% ##### ( 3, 4 ] 128987 10.502% 86.593% ## ( 4, 6 ] 123904 10.088% 96.682% ## ( 6, 10 ] 29660 2.415% 99.096% ( 10, 15 ] 4466 0.364% 99.460% ( 15, 22 ] 554 0.045% 99.505% ( 22, 34 ] 68 0.006% 99.511% ( 34, 51 ] 58 0.005% 99.515% ( 51, 76 ] 82 0.007% 99.522% ( 76, 110 ] 151 0.012% 99.534% ( 110, 170 ] 533 0.043% 99.578% ( 170, 250 ] 449 0.037% 99.614% ( 250, 380 ] 596 0.049% 99.663% ( 380, 580 ] 1037 0.084% 99.747% ( 580, 870 ] 1363 0.111% 99.858% ( 870, 1300 ] 1433 0.117% 99.975% ( 1300, 1900 ] 954 0.078% 100.053% ( 1900, 2900 ] 769 0.063% 100.115% ( 2900, 4400 ] 677 0.055% 100.170% ( 4400, 6600 ] 170 0.014% 100.184% ( 6600, 9900 ] 41 0.003% 100.188% ( 9900, 14000 ] 7 0.001% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 33000, 50000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 620469 Average: 20.9638 StdDev: 233.39 Min: 0 Median: 1.9532 Max: 59938 Percentiles: P50: 1.95 P75: 2.98 P99: 422.67 P99.9: 1860.40 P99.99: 2772.33 ------------------------------------------------------ [ 0, 1 ] 12668 2.042% 2.042% ( 1, 2 ] 312175 50.313% 52.354% ########## ( 2, 3 ] 142748 23.006% 75.361% ##### ( 3, 4 ] 61710 9.946% 85.307% ## ( 4, 6 ] 57927 9.336% 94.643% ## ( 6, 10 ] 14740 2.376% 97.018% ( 10, 15 ] 1868 0.301% 97.319% ( 15, 22 ] 244 0.039% 97.359% ( 22, 34 ] 31 0.005% 97.364% ( 34, 51 ] 28 0.005% 97.368% ( 51, 76 ] 48 0.008% 97.376% ( 76, 110 ] 1314 0.212% 97.588% ( 110, 170 ] 4366 0.704% 98.291% ( 170, 250 ] 2376 0.383% 98.674% ( 250, 380 ] 1607 0.259% 98.933% ( 380, 580 ] 1942 0.313% 99.246% ( 580, 870 ] 1854 0.299% 99.545% ( 870, 1300 ] 1460 0.235% 99.780% ( 1300, 1900 ] 795 0.128% 99.908% ( 1900, 2900 ] 580 0.093% 100.002% ( 2900, 4400 ] 526 0.085% 100.087% ( 4400, 6600 ] 150 0.024% 100.111% ( 6600, 9900 ] 27 0.004% 100.115% ( 9900, 14000 ] 1 0.000% 100.115% ( 14000, 22000 ] 3 0.000% 100.116% ( 22000, 33000 ] 1 0.000% 100.116% ( 33000, 50000 ] 4 0.001% 100.117% ( 50000, 75000 ] 1 0.000% 100.117% ** Level 5 read latency histogram (micros): Count: 91545 Average: 495.7091 StdDev: 842.68 Min: 73 Median: 204.9106 Max: 56951 Percentiles: P50: 204.91 P75: 501.11 P99: 4321.53 P99.9: 7120.42 P99.99: 9894.80 ------------------------------------------------------ ( 51, 76 ] 12 0.013% 0.013% ( 76, 110 ] 9253 10.108% 10.121% ## ( 110, 170 ] 29991 32.761% 42.882% ####### ( 170, 250 ] 14933 16.312% 59.194% ### ( 250, 380 ] 8934 9.759% 68.953% ## ( 380, 580 ] 9142 9.986% 78.939% ## ( 580, 870 ] 7247 7.916% 86.856% ## ( 870, 1300 ] 4767 5.207% 92.063% # ( 1300, 1900 ] 2605 2.846% 94.909% # ( 1900, 2900 ] 1962 2.143% 97.052% ( 2900, 4400 ] 1882 2.056% 99.108% ( 4400, 6600 ] 710 0.776% 99.883% ( 6600, 9900 ] 98 0.107% 99.990% ( 9900, 14000 ] 5 0.005% 99.996% ( 14000, 22000 ] 3 0.003% 99.999% ( 50000, 75000 ] 2 0.002% 100.001% ** DB Stats ** Uptime(secs): 16.5 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 178.41 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.421 H:M:S, 32.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 263.41 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.250 H:M:S, 6.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/30-15:49:46 ... thread 0: (1000000,4000000) ops and (235700.5,193738.5) ops/second in (4.242672,20.646385) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/15 3.75 GB 100.7 3.3 0.0 3.3 7.1 3.7 0.0 1.9 100.7 214.3 33.79 23.68 67 0.504 3454K 3707 0.0 0.0 L1 19/19 1.37 GB 0.0 1.4 0.9 0.4 1.4 0.9 0.0 1.5 91.8 91.3 15.41 7.39 1 15.413 1437K 8101 0.0 0.0 L3 16/0 890.68 MB 0.9 3.3 2.0 1.3 3.2 1.9 0.0 1.6 100.6 99.8 33.18 15.45 16 2.074 1724K 3246 0.0 0.0 L4 159/0 9.45 GB 1.0 6.0 3.7 2.3 5.9 3.6 0.3 1.6 118.0 116.6 52.12 28.08 43 1.212 2556K 7 0.0 0.0 L5 1128/0 93.26 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1341/34 108.70 GB 0.0 14.0 6.6 7.4 17.6 10.2 4.3 4.7 106.3 134.1 134.51 74.60 127 1.059 9172K 15K 0.0 0.0 Int 0/0 0.00 KB 0.0 14.0 6.6 7.4 17.6 10.2 4.3 4.7 106.3 134.1 134.51 74.60 127 1.059 9172K 15K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 14.0 6.6 7.4 13.9 6.5 0.0 0.0 122.7 121.8 116.57 64.01 72 1.619 9172K 15K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 214.3 17.93 10.59 55 0.326 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.8 total, 20.6 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.61 GB write, 867.49 MB/s write, 13.97 GB read, 687.97 MB/s read, 134.5 seconds Interval compaction: 17.61 GB write, 873.52 MB/s write, 13.97 GB read, 692.76 MB/s read, 134.5 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 5, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 5, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 5, total-stops: 42, interval: 47 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1202911 Average: 13.6836 StdDev: 115.44 Min: 0 Median: 1.6527 Max: 35768 Percentiles: P50: 1.65 P75: 2.41 P99: 248.54 P99.9: 1035.18 P99.99: 1740.20 ------------------------------------------------------ [ 0, 1 ] 176438 14.668% 14.668% ### ( 1, 2 ] 651135 54.130% 68.798% ########### ( 2, 3 ] 180251 14.985% 83.782% ### ( 3, 4 ] 77238 6.421% 90.203% # ( 4, 6 ] 59865 4.977% 95.180% # ( 6, 10 ] 13349 1.110% 96.289% ( 10, 15 ] 2163 0.180% 96.469% ( 15, 22 ] 321 0.027% 96.496% ( 22, 34 ] 39 0.003% 96.499% ( 34, 51 ] 30 0.002% 96.502% ( 51, 76 ] 40 0.003% 96.505% ( 76, 110 ] 4683 0.389% 96.894% ( 110, 170 ] 17104 1.422% 98.316% ( 170, 250 ] 8381 0.697% 99.013% ( 250, 380 ] 4156 0.345% 99.358% ( 380, 580 ] 3813 0.317% 99.675% ( 580, 870 ] 2275 0.189% 99.864% ( 870, 1300 ] 1117 0.093% 99.957% ( 1300, 1900 ] 538 0.045% 100.002% ( 1900, 2900 ] 361 0.030% 100.032% ( 2900, 4400 ] 304 0.025% 100.057% ( 4400, 6600 ] 80 0.007% 100.064% ( 6600, 9900 ] 7 0.001% 100.065% ( 9900, 14000 ] 1 0.000% 100.065% ( 14000, 22000 ] 1 0.000% 100.065% ( 22000, 33000 ] 1 0.000% 100.065% ( 33000, 50000 ] 1 0.000% 100.065% ** Level 1 read latency histogram (micros): Count: 139174 Average: 11.4114 StdDev: 169.51 Min: 1 Median: 2.4774 Max: 50375 Percentiles: P50: 2.48 P75: 3.82 P99: 169.13 P99.9: 1395.09 P99.99: 4415.14 ------------------------------------------------------ [ 0, 1 ] 3019 2.169% 2.169% ( 1, 2 ] 48838 35.091% 37.261% ####### ( 2, 3 ] 37138 26.685% 63.945% ##### ( 3, 4 ] 18801 13.509% 77.454% ### ( 4, 6 ] 21791 15.657% 93.112% ### ( 6, 10 ] 5370 3.858% 96.970% # ( 10, 15 ] 964 0.693% 97.663% ( 15, 22 ] 392 0.282% 97.944% ( 22, 34 ] 53 0.038% 97.982% ( 34, 51 ] 6 0.004% 97.987% ( 51, 76 ] 6 0.004% 97.991% ( 76, 110 ] 335 0.241% 98.232% ( 110, 170 ] 1085 0.780% 99.011% ( 170, 250 ] 458 0.329% 99.340% ( 250, 380 ] 247 0.177% 99.518% ( 380, 580 ] 244 0.175% 99.693% ( 580, 870 ] 167 0.120% 99.813% ( 870, 1300 ] 111 0.080% 99.893% ( 1300, 1900 ] 62 0.045% 99.937% ( 1900, 2900 ] 53 0.038% 99.976% ( 2900, 4400 ] 20 0.014% 99.990% ( 4400, 6600 ] 12 0.009% 99.999% ( 6600, 9900 ] 2 0.001% 100.000% ( 50000, 75000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 378812 Average: 10.4908 StdDev: 135.91 Min: 1 Median: 1.9570 Max: 12423 Percentiles: P50: 1.96 P75: 2.97 P99: 11.39 P99.9: 1384.58 P99.99: 2403.05 ------------------------------------------------------ [ 0, 1 ] 11657 3.077% 3.077% # ( 1, 2 ] 185730 49.030% 52.107% ########## ( 2, 3 ] 89808 23.708% 75.815% ##### ( 3, 4 ] 38234 10.093% 85.908% ## ( 4, 6 ] 37842 9.990% 95.897% ## ( 6, 10 ] 11207 2.958% 98.856% # ( 10, 15 ] 1964 0.518% 99.374% ( 15, 22 ] 310 0.082% 99.456% ( 22, 34 ] 81 0.021% 99.478% ( 34, 51 ] 33 0.009% 99.486% ( 51, 76 ] 21 0.006% 99.492% ( 76, 110 ] 30 0.008% 99.500% ( 110, 170 ] 100 0.026% 99.526% ( 170, 250 ] 105 0.028% 99.554% ( 250, 380 ] 203 0.054% 99.607% ( 380, 580 ] 313 0.083% 99.690% ( 580, 870 ] 386 0.102% 99.792% ( 870, 1300 ] 370 0.098% 99.890% ( 1300, 1900 ] 278 0.073% 99.963% ( 1900, 2900 ] 203 0.054% 100.017% ( 2900, 4400 ] 176 0.046% 100.063% ( 4400, 6600 ] 62 0.016% 100.079% ( 6600, 9900 ] 8 0.002% 100.082% ( 9900, 14000 ] 1 0.000% 100.082% ** Level 3 read latency histogram (micros): Count: 1229179 Average: 11.4879 StdDev: 150.45 Min: 1 Median: 1.9670 Max: 42082 Percentiles: P50: 1.97 P75: 2.96 P99: 9.96 P99.9: 1023.28 P99.99: 1415.77 ------------------------------------------------------ [ 0, 1 ] 23761 1.933% 1.933% ( 1, 2 ] 611000 49.708% 51.641% ########## ( 2, 3 ] 299835 24.393% 76.034% ##### ( 3, 4 ] 129003 10.495% 86.529% ## ( 4, 6 ] 123943 10.083% 96.613% ## ( 6, 10 ] 29662 2.413% 99.026% ( 10, 15 ] 4467 0.363% 99.389% ( 15, 22 ] 554 0.045% 99.434% ( 22, 34 ] 68 0.006% 99.440% ( 34, 51 ] 58 0.005% 99.445% ( 51, 76 ] 82 0.007% 99.451% ( 76, 110 ] 305 0.025% 99.476% ( 110, 170 ] 987 0.080% 99.556% ( 170, 250 ] 606 0.049% 99.606% ( 250, 380 ] 646 0.053% 99.658% ( 380, 580 ] 1085 0.088% 99.746% ( 580, 870 ] 1377 0.112% 99.858% ( 870, 1300 ] 1433 0.117% 99.975% ( 1300, 1900 ] 954 0.078% 100.053% ( 1900, 2900 ] 769 0.063% 100.115% ( 2900, 4400 ] 677 0.055% 100.170% ( 4400, 6600 ] 170 0.014% 100.184% ( 6600, 9900 ] 41 0.003% 100.187% ( 9900, 14000 ] 7 0.001% 100.188% ( 14000, 22000 ] 1 0.000% 100.188% ( 33000, 50000 ] 1 0.000% 100.188% ** Level 4 read latency histogram (micros): Count: 630096 Average: 23.4113 StdDev: 233.06 Min: 0 Median: 1.9685 Max: 59938 Percentiles: P50: 1.97 P75: 3.08 P99: 462.93 P99.9: 1858.63 P99.99: 2775.20 ------------------------------------------------------ [ 0, 1 ] 12668 2.010% 2.010% ( 1, 2 ] 312199 49.548% 51.558% ########## ( 2, 3 ] 142816 22.666% 74.224% ##### ( 3, 4 ] 61765 9.802% 84.027% ## ( 4, 6 ] 57982 9.202% 93.229% ## ( 6, 10 ] 14748 2.341% 95.569% ( 10, 15 ] 1869 0.297% 95.866% ( 15, 22 ] 244 0.039% 95.905% ( 22, 34 ] 31 0.005% 95.910% ( 34, 51 ] 28 0.004% 95.914% ( 51, 76 ] 50 0.008% 95.922% ( 76, 110 ] 2928 0.465% 96.387% ( 110, 170 ] 9096 1.444% 97.830% ( 170, 250 ] 4112 0.653% 98.483% ( 250, 380 ] 2234 0.355% 98.837% ( 380, 580 ] 2472 0.392% 99.230% ( 580, 870 ] 2002 0.318% 99.547% ( 870, 1300 ] 1478 0.235% 99.782% ( 1300, 1900 ] 799 0.127% 99.909% ( 1900, 2900 ] 585 0.093% 100.002% ( 2900, 4400 ] 528 0.084% 100.085% ( 4400, 6600 ] 150 0.024% 100.109% ( 6600, 9900 ] 27 0.004% 100.113% ( 9900, 14000 ] 1 0.000% 100.114% ( 14000, 22000 ] 3 0.000% 100.114% ( 22000, 33000 ] 1 0.000% 100.114% ( 33000, 50000 ] 4 0.001% 100.115% ( 50000, 75000 ] 1 0.000% 100.115% ** Level 5 read latency histogram (micros): Count: 150560 Average: 373.5870 StdDev: 679.71 Min: 73 Median: 165.7922 Max: 56951 Percentiles: P50: 165.79 P75: 330.53 P99: 3852.91 P99.9: 6465.03 P99.99: 9696.07 ------------------------------------------------------ ( 51, 76 ] 20 0.013% 0.013% ( 76, 110 ] 19704 13.087% 13.100% ### ( 110, 170 ] 59746 39.683% 52.783% ######## ( 170, 250 ] 25458 16.909% 69.692% ### ( 250, 380 ] 12901 8.569% 78.260% ## ( 380, 580 ] 12351 8.203% 86.464% ## ( 580, 870 ] 8139 5.406% 91.870% # ( 870, 1300 ] 4916 3.265% 95.135% # ( 1300, 1900 ] 2627 1.745% 96.880% ( 1900, 2900 ] 1993 1.324% 98.203% ( 2900, 4400 ] 1888 1.254% 99.457% ( 4400, 6600 ] 710 0.472% 99.929% ( 6600, 9900 ] 98 0.065% 99.994% ( 9900, 14000 ] 5 0.003% 99.997% ( 14000, 22000 ] 3 0.002% 99.999% ( 50000, 75000 ] 2 0.001% 100.001% ** DB Stats ** Uptime(secs): 20.8 total, 4.2 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 189.34 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.163 H:M:S, 29.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 231.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.742 H:M:S, 17.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 239.787 micros/op 16675 ops/sec 60.149 seconds 1002996 operations; 16.2 MB/s (246999 of 246999 found) Microseconds per read: Count: 1002996 Average: 239.7887 StdDev: 670.95 Min: 2 Median: 171.6857 Max: 82243 Percentiles: P50: 171.69 P75: 233.80 P99: 1145.89 P99.9: 10777.37 P99.99: 27777.20 ------------------------------------------------------ ( 1, 2 ] 11 0.001% 0.001% ( 2, 3 ] 34 0.003% 0.004% ( 3, 4 ] 51 0.005% 0.010% ( 4, 6 ] 91 0.009% 0.019% ( 6, 10 ] 498 0.050% 0.068% ( 10, 15 ] 3342 0.333% 0.401% ( 15, 22 ] 2864 0.286% 0.687% ( 22, 34 ] 7177 0.716% 1.403% ( 34, 51 ] 2135 0.213% 1.615% ( 51, 76 ] 378 0.038% 1.653% ( 76, 110 ] 24282 2.421% 4.074% ( 110, 170 ] 453830 45.247% 49.322% ######### ( 170, 250 ] 322946 32.198% 81.520% ###### ( 250, 380 ] 112076 11.174% 92.694% ## ( 380, 580 ] 42501 4.237% 96.931% # ( 580, 870 ] 16450 1.640% 98.571% ( 870, 1300 ] 6702 0.668% 99.239% ( 1300, 1900 ] 2782 0.277% 99.517% ( 1900, 2900 ] 1808 0.180% 99.697% ( 2900, 4400 ] 1301 0.130% 99.827% ( 4400, 6600 ] 406 0.040% 99.867% ( 6600, 9900 ] 227 0.023% 99.890% ( 9900, 14000 ] 472 0.047% 99.937% ( 14000, 22000 ] 465 0.046% 99.983% ( 22000, 33000 ] 127 0.013% 99.996% ( 33000, 50000 ] 25 0.002% 99.999% ( 50000, 75000 ] 14 0.001% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 17535480 rocksdb.bloom.filter.full.positive COUNT : 1172817 rocksdb.bloom.filter.full.true.positive COUNT : 1002534 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 462 rocksdb.memtable.miss COUNT : 1002534 rocksdb.l0.hit COUNT : 43173 rocksdb.l1.hit COUNT : 12661 rocksdb.l2andup.hit COUNT : 946700 rocksdb.compaction.key.drop.new COUNT : 152561 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4873878 rocksdb.number.keys.read COUNT : 1002996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 5029842096 rocksdb.bytes.read COUNT : 1002996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1575 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 41515723 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 19527997682 rocksdb.compact.write.bytes COUNT : 19374495178 rocksdb.flush.write.bytes COUNT : 5020466120 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 27 rocksdb.number.superversion_acquires COUNT : 934 rocksdb.number.superversion_releases COUNT : 908 rocksdb.number.superversion_cleanups COUNT : 237 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 199 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4979852288 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1815552 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 24290982784 rocksdb.non.last.level.read.count COUNT : 5726951 rocksdb.block.checksum.compute.count COUNT : 5915353 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 170.055702 P95 : 487.714035 P99 : 1144.762341 P100 : 82241.000000 COUNT : 1002996 SUM : 239675028 rocksdb.db.write.micros P50 : 1.557713 P95 : 4.748073 P99 : 10.354349 P100 : 967331.000000 COUNT : 4873878 SUM : 53915445 rocksdb.compaction.times.micros P50 : 1195142.857143 P95 : 2829166.666667 P99 : 15412930.000000 P100 : 15412930.000000 COUNT : 77 SUM : 126174558 rocksdb.compaction.times.cpu_micros P50 : 813863.636364 P95 : 1770500.000000 P99 : 7259000.000000 P100 : 7386208.000000 COUNT : 77 SUM : 73411777 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 783.000000 P95 : 140000.000000 P99 : 289000.000000 P100 : 289096.000000 COUNT : 70 SUM : 1224680 rocksdb.compaction.outfile.sync.micros P50 : 70394.736842 P95 : 318033.333333 P99 : 548150.000000 P100 : 967635.000000 COUNT : 223 SUM : 22912716 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 220.000000 P95 : 4936.250000 P99 : 7243.500000 P100 : 9434.000000 COUNT : 161 SUM : 160036 rocksdb.table.open.io.micros P50 : 8769.817073 P95 : 19616.822430 P99 : 21971.962617 P100 : 28203.000000 COUNT : 1575 SUM : 14131696 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.911749 P95 : 82.994989 P99 : 201.793369 P100 : 59941.000000 COUNT : 4730048 SUM : 80214185 rocksdb.read.block.get.micros P50 : 139.417554 P95 : 353.512277 P99 : 1049.149778 P100 : 56954.000000 COUNT : 1183723 SUM : 204454277 rocksdb.write.raw.block.micros P50 : 0.535723 P95 : 1.364269 P99 : 3.849502 P100 : 65839.000000 COUNT : 5908423 SUM : 17114216 rocksdb.numfiles.in.singlecompaction P50 : 2.634615 P95 : 3.980303 P99 : 22.840000 P100 : 30.000000 COUNT : 93 SUM : 322 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.572752 P95 : 1929.938520 P99 : 2707.116509 P100 : 967148.000000 COUNT : 192779 SUM : 41534574 rocksdb.sst.read.micros P50 : 2.197329 P95 : 168.382404 P99 : 484.976278 P100 : 59938.000000 COUNT : 5726983 SUM : 269665967 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1002996 SUM : 1002996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4873878 SUM : 5029842096 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 220370.370370 P95 : 987500.000000 P99 : 1200390.000000 P100 : 1200390.000000 COUNT : 70 SUM : 20970568 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 247 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0