[2024-12-07 09:46:40.519773] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-12-07 09:46:40.520038] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid968448 ] Set seed to 1733561200513104 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-07 09:46:40.620118] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-07 09:46:40.661950] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-07 09:46:40.661997] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 968448' to capture a snapshot of events at runtime. [2024-12-07 09:46:40.662008] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-07 09:46:40.662019] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-07 09:46:40.662027] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid968448 for offline analysis/debug. [2024-12-07 09:46:40.662134] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sat Dec 7 09:46:44 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/07-09:46:52 ... thread 0: (1000000,1000000) ops and (124653.2,124653.2) ops/second in (8.022258,8.022258) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/10 1.58 GB 44.0 0.4 0.0 0.4 1.2 0.8 0.0 1.5 36.6 115.9 10.32 4.11 10 1.032 383K 541 0.0 0.0 L1 4/4 413.12 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 24/10 2.19 GB 1.3 2.3 1.4 0.9 2.2 1.4 0.3 1.6 95.7 94.7 24.26 11.01 8 3.032 1046K 0 0.0 0.0 L4 219/28 12.45 GB 1.1 3.8 2.8 1.1 3.8 2.7 0.0 1.4 108.7 107.7 36.14 18.53 30 1.205 2212K 550 0.0 0.0 L5 1076/0 88.95 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1335/52 105.58 GB 0.0 6.5 4.1 2.3 7.2 4.9 0.3 9.0 93.7 104.5 70.72 33.65 48 1.473 3643K 1091 0.0 0.0 Int 0/0 0.00 KB 0.0 6.5 4.1 2.3 7.2 4.9 0.3 9.0 93.7 104.5 70.72 33.65 48 1.473 3643K 1091 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.5 4.1 2.3 6.4 4.1 0.0 0.0 104.7 103.8 63.30 31.17 39 1.623 3643K 1091 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 110.3 7.42 2.48 9 0.825 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.2 total, 8.0 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.21 GB write, 901.00 MB/s write, 6.47 GB read, 808.56 MB/s read, 70.7 seconds Interval compaction: 7.21 GB write, 920.62 MB/s write, 6.47 GB read, 826.16 MB/s read, 70.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 29, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 305997 Average: 12.9919 StdDev: 191.87 Min: 1 Median: 1.8939 Max: 47247 Percentiles: P50: 1.89 P75: 2.88 P99: 11.48 P99.9: 2335.19 P99.99: 4207.39 ------------------------------------------------------ [ 0, 1 ] 10747 3.512% 3.512% # ( 1, 2 ] 159139 52.007% 55.519% ########## ( 2, 3 ] 67654 22.109% 77.628% #### ( 3, 4 ] 28820 9.418% 87.047% ## ( 4, 6 ] 29068 9.499% 96.546% ## ( 6, 10 ] 7196 2.352% 98.898% ( 10, 15 ] 1067 0.349% 99.246% ( 15, 22 ] 128 0.042% 99.288% ( 22, 34 ] 16 0.005% 99.293% ( 34, 51 ] 10 0.003% 99.297% ( 51, 76 ] 14 0.005% 99.301% ( 76, 110 ] 13 0.004% 99.306% ( 110, 170 ] 35 0.011% 99.317% ( 170, 250 ] 44 0.014% 99.331% ( 250, 380 ] 150 0.049% 99.380% ( 380, 580 ] 260 0.085% 99.465% ( 580, 870 ] 485 0.158% 99.624% ( 870, 1300 ] 455 0.149% 99.773% ( 1300, 1900 ] 298 0.097% 99.870% ( 1900, 2900 ] 216 0.071% 99.941% ( 2900, 4400 ] 176 0.058% 99.998% ( 4400, 6600 ] 43 0.014% 100.012% ( 6600, 9900 ] 13 0.004% 100.016% ( 9900, 14000 ] 1 0.000% 100.017% ( 14000, 22000 ] 5 0.002% 100.018% ( 22000, 33000 ] 1 0.000% 100.019% ( 33000, 50000 ] 1 0.000% 100.019% ** Level 1 read latency histogram (micros): Count: 26827 Average: 27.7903 StdDev: 382.08 Min: 1 Median: 2.0457 Max: 33950 Percentiles: P50: 2.05 P75: 3.13 P99: 600.41 P99.9: 4337.72 P99.99: 16538.40 ------------------------------------------------------ [ 0, 1 ] 346 1.290% 1.290% ( 1, 2 ] 12764 47.579% 48.869% ########## ( 2, 3 ] 6634 24.729% 73.597% ##### ( 3, 4 ] 2879 10.732% 84.329% ## ( 4, 6 ] 3027 11.283% 95.613% ## ( 6, 10 ] 728 2.714% 98.326% # ( 10, 15 ] 116 0.432% 98.759% ( 15, 22 ] 13 0.048% 98.807% ( 22, 34 ] 1 0.004% 98.811% ( 51, 76 ] 6 0.022% 98.833% ( 76, 110 ] 2 0.007% 98.841% ( 110, 170 ] 1 0.004% 98.844% ( 170, 250 ] 3 0.011% 98.856% ( 250, 380 ] 13 0.048% 98.904% ( 380, 580 ] 22 0.082% 98.986% ( 580, 870 ] 53 0.198% 99.184% ( 870, 1300 ] 59 0.220% 99.404% ( 1300, 1900 ] 46 0.171% 99.575% ( 1900, 2900 ] 45 0.168% 99.743% ( 2900, 4400 ] 44 0.164% 99.907% ( 4400, 6600 ] 15 0.056% 99.963% ( 6600, 9900 ] 6 0.022% 99.985% ( 9900, 14000 ] 1 0.004% 99.989% ( 14000, 22000 ] 1 0.004% 99.993% ( 22000, 33000 ] 1 0.004% 99.996% ( 33000, 50000 ] 1 0.004% 100.000% ** Level 2 read latency histogram (micros): Count: 233320 Average: 14.7402 StdDev: 207.12 Min: 1 Median: 1.9406 Max: 35035 Percentiles: P50: 1.94 P75: 2.96 P99: 12.80 P99.9: 2261.05 P99.99: 3452.03 ------------------------------------------------------ [ 0, 1 ] 7254 3.109% 3.109% # ( 1, 2 ] 116316 49.853% 52.962% ########## ( 2, 3 ] 53297 22.843% 75.804% ##### ( 3, 4 ] 23538 10.088% 85.893% ## ( 4, 6 ] 23235 9.958% 95.851% ## ( 6, 10 ] 6651 2.851% 98.702% # ( 10, 15 ] 1243 0.533% 99.235% ( 15, 22 ] 187 0.080% 99.315% ( 22, 34 ] 34 0.015% 99.329% ( 34, 51 ] 21 0.009% 99.338% ( 51, 76 ] 16 0.007% 99.345% ( 76, 110 ] 16 0.007% 99.352% ( 110, 170 ] 39 0.017% 99.369% ( 170, 250 ] 57 0.024% 99.393% ( 250, 380 ] 111 0.048% 99.441% ( 380, 580 ] 179 0.077% 99.517% ( 580, 870 ] 241 0.103% 99.621% ( 870, 1300 ] 308 0.132% 99.753% ( 1300, 1900 ] 261 0.112% 99.865% ( 1900, 2900 ] 229 0.098% 99.963% ( 2900, 4400 ] 173 0.074% 100.037% ( 4400, 6600 ] 59 0.025% 100.062% ( 6600, 9900 ] 12 0.005% 100.067% ( 9900, 14000 ] 2 0.001% 100.068% ( 14000, 22000 ] 6 0.003% 100.071% ( 33000, 50000 ] 1 0.000% 100.071% ** Level 3 read latency histogram (micros): Count: 783728 Average: 11.8598 StdDev: 196.23 Min: 1 Median: 1.9543 Max: 48102 Percentiles: P50: 1.95 P75: 2.95 P99: 9.68 P99.9: 977.51 P99.99: 1387.71 ------------------------------------------------------ [ 0, 1 ] 13436 1.714% 1.714% ( 1, 2 ] 396569 50.600% 52.315% ########## ( 2, 3 ] 186290 23.770% 76.084% ##### ( 3, 4 ] 82165 10.484% 86.568% ## ( 4, 6 ] 80337 10.251% 96.819% ## ( 6, 10 ] 18587 2.372% 99.191% ( 10, 15 ] 2913 0.372% 99.562% ( 15, 22 ] 395 0.050% 99.613% ( 22, 34 ] 51 0.007% 99.619% ( 34, 51 ] 48 0.006% 99.625% ( 51, 76 ] 62 0.008% 99.633% ( 76, 110 ] 14 0.002% 99.635% ( 110, 170 ] 54 0.007% 99.642% ( 170, 250 ] 131 0.017% 99.659% ( 250, 380 ] 258 0.033% 99.691% ( 380, 580 ] 555 0.071% 99.762% ( 580, 870 ] 870 0.111% 99.873% ( 870, 1300 ] 837 0.107% 99.980% ( 1300, 1900 ] 531 0.068% 100.048% ( 1900, 2900 ] 428 0.055% 100.102% ( 2900, 4400 ] 499 0.064% 100.166% ( 4400, 6600 ] 158 0.020% 100.186% ( 6600, 9900 ] 31 0.004% 100.190% ( 9900, 14000 ] 5 0.001% 100.191% ( 14000, 22000 ] 8 0.001% 100.192% ( 22000, 33000 ] 6 0.001% 100.193% ( 33000, 50000 ] 3 0.000% 100.193% ** Level 4 read latency histogram (micros): Count: 408191 Average: 23.0790 StdDev: 246.11 Min: 1 Median: 1.9257 Max: 48929 Percentiles: P50: 1.93 P75: 2.92 P99: 487.29 P99.9: 2500.75 P99.99: 3288.40 ------------------------------------------------------ [ 0, 1 ] 8102 1.985% 1.985% ( 1, 2 ] 211729 51.870% 53.855% ########## ( 2, 3 ] 93907 23.006% 76.861% ##### ( 3, 4 ] 41388 10.139% 87.000% ## ( 4, 6 ] 36665 8.982% 95.982% ## ( 6, 10 ] 9678 2.371% 98.353% ( 10, 15 ] 1445 0.354% 98.707% ( 15, 22 ] 179 0.044% 98.751% ( 22, 34 ] 17 0.004% 98.755% ( 34, 51 ] 22 0.005% 98.761% ( 51, 76 ] 36 0.009% 98.769% ( 76, 110 ] 24 0.006% 98.775% ( 110, 170 ] 68 0.017% 98.792% ( 170, 250 ] 157 0.038% 98.830% ( 250, 380 ] 325 0.080% 98.910% ( 380, 580 ] 688 0.169% 99.079% ( 580, 870 ] 1046 0.256% 99.335% ( 870, 1300 ] 1172 0.287% 99.622% ( 1300, 1900 ] 810 0.198% 99.820% ( 1900, 2900 ] 544 0.133% 99.954% ( 2900, 4400 ] 580 0.142% 100.096% ( 4400, 6600 ] 160 0.039% 100.135% ( 6600, 9900 ] 28 0.007% 100.142% ( 9900, 14000 ] 6 0.001% 100.143% ( 22000, 33000 ] 2 0.000% 100.144% ( 33000, 50000 ] 1 0.000% 100.144% ** Level 5 read latency histogram (micros): Count: 13483 Average: 1689.6489 StdDev: 1931.09 Min: 82 Median: 1114.5835 Max: 58429 Percentiles: P50: 1114.58 P75: 2239.39 P99: 7113.68 P99.9: 16427.20 P99.99: 49154.13 ------------------------------------------------------ ( 76, 110 ] 13 0.096% 0.096% ( 110, 170 ] 67 0.497% 0.593% ( 170, 250 ] 232 1.721% 2.314% ( 250, 380 ] 740 5.488% 7.802% # ( 380, 580 ] 1594 11.822% 19.625% ## ( 580, 870 ] 2566 19.031% 38.656% #### ( 870, 1300 ] 2689 19.944% 58.600% #### ( 1300, 1900 ] 1712 12.697% 71.297% ### ( 1900, 2900 ] 1471 10.910% 82.207% ## ( 2900, 4400 ] 1529 11.340% 93.547% ## ( 4400, 6600 ] 714 5.296% 98.843% # ( 6600, 9900 ] 136 1.009% 99.852% ( 9900, 14000 ] 5 0.037% 99.889% ( 14000, 22000 ] 5 0.037% 99.926% ( 22000, 33000 ] 2 0.015% 99.941% ( 33000, 50000 ] 7 0.052% 99.993% ( 50000, 75000 ] 2 0.015% 100.007% ** DB Stats ** Uptime(secs): 8.2 total, 8.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 120.04 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.988 H:M:S, 48.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 122.64 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.988 H:M:S, 49.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/07-09:46:58 ... thread 0: (1000000,2000000) ops and (194176.3,151834.7) ops/second in (5.149960,13.172218) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/8 1.84 GB 138.4 1.4 0.0 1.4 3.2 1.8 0.0 1.7 67.5 159.4 20.54 11.27 28 0.734 1407K 1848 0.0 0.0 L1 15/0 1.18 GB 121.0 1.2 0.8 0.4 1.2 0.8 0.0 1.5 93.5 93.0 13.01 6.42 1 13.013 1235K 5936 0.0 0.0 L3 12/0 997.25 MB 1.0 2.3 1.4 0.9 2.2 1.4 0.3 1.6 95.7 94.7 24.26 11.01 8 3.032 1046K 0 0.0 0.0 L4 178/0 9.49 GB 1.0 6.8 4.0 2.8 6.7 3.9 0.0 1.7 114.4 112.8 61.03 32.87 40 1.526 3344K 12K 0.0 0.0 L5 1132/0 93.07 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1349/8 106.56 GB 0.0 11.6 6.1 5.5 13.3 7.9 4.4 7.2 100.2 115.0 118.85 61.57 77 1.543 7032K 20K 0.0 0.0 Int 0/0 0.00 KB 0.0 11.6 6.1 5.5 13.3 7.9 4.4 7.2 100.2 115.0 118.85 61.57 77 1.543 7032K 20K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.6 6.1 5.5 11.5 6.0 0.0 0.0 111.8 110.6 106.52 55.99 53 2.010 7032K 20K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 153.3 12.33 5.58 24 0.514 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 13.3 total, 13.2 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.35 GB write, 1023.86 MB/s write, 11.63 GB read, 891.80 MB/s read, 118.8 seconds Interval compaction: 13.35 GB write, 1037.44 MB/s write, 11.63 GB read, 903.62 MB/s read, 118.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 42, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 632893 Average: 14.6936 StdDev: 159.62 Min: 1 Median: 1.7395 Max: 47247 Percentiles: P50: 1.74 P75: 2.59 P99: 303.26 P99.9: 1711.42 P99.99: 3321.83 ------------------------------------------------------ [ 0, 1 ] 56188 8.878% 8.878% ## ( 1, 2 ] 351923 55.605% 64.483% ########### ( 2, 3 ] 113192 17.885% 82.368% #### ( 3, 4 ] 44500 7.031% 89.399% # ( 4, 6 ] 41412 6.543% 95.943% # ( 6, 10 ] 9902 1.565% 97.507% ( 10, 15 ] 1589 0.251% 97.758% ( 15, 22 ] 208 0.033% 97.791% ( 22, 34 ] 20 0.003% 97.794% ( 34, 51 ] 13 0.002% 97.796% ( 51, 76 ] 22 0.003% 97.800% ( 76, 110 ] 936 0.148% 97.948% ( 110, 170 ] 3501 0.553% 98.501% ( 170, 250 ] 2437 0.385% 98.886% ( 250, 380 ] 1760 0.278% 99.164% ( 380, 580 ] 1806 0.285% 99.450% ( 580, 870 ] 1516 0.240% 99.689% ( 870, 1300 ] 995 0.157% 99.846% ( 1300, 1900 ] 496 0.078% 99.925% ( 1900, 2900 ] 342 0.054% 99.979% ( 2900, 4400 ] 255 0.040% 100.019% ( 4400, 6600 ] 67 0.011% 100.030% ( 6600, 9900 ] 14 0.002% 100.032% ( 9900, 14000 ] 1 0.000% 100.032% ( 14000, 22000 ] 6 0.001% 100.033% ( 22000, 33000 ] 1 0.000% 100.033% ( 33000, 50000 ] 1 0.000% 100.033% ** Level 1 read latency histogram (micros): Count: 105306 Average: 12.3543 StdDev: 206.51 Min: 1 Median: 2.5569 Max: 33950 Percentiles: P50: 2.56 P75: 3.83 P99: 15.88 P99.9: 2573.88 P99.99: 6793.63 ------------------------------------------------------ [ 0, 1 ] 867 0.823% 0.823% ( 1, 2 ] 34437 32.702% 33.525% ####### ( 2, 3 ] 31153 29.583% 63.108% ###### ( 3, 4 ] 15024 14.267% 77.375% ### ( 4, 6 ] 17086 16.225% 93.601% ### ( 6, 10 ] 4791 4.550% 98.150% # ( 10, 15 ] 861 0.818% 98.968% ( 15, 22 ] 269 0.255% 99.223% ( 22, 34 ] 20 0.019% 99.242% ( 34, 51 ] 4 0.004% 99.246% ( 51, 76 ] 18 0.017% 99.263% ( 76, 110 ] 46 0.044% 99.307% ( 110, 170 ] 114 0.108% 99.415% ( 170, 250 ] 73 0.069% 99.484% ( 250, 380 ] 73 0.069% 99.554% ( 380, 580 ] 84 0.080% 99.633% ( 580, 870 ] 104 0.099% 99.732% ( 870, 1300 ] 82 0.078% 99.810% ( 1300, 1900 ] 61 0.058% 99.868% ( 1900, 2900 ] 50 0.047% 99.915% ( 2900, 4400 ] 57 0.054% 99.970% ( 4400, 6600 ] 21 0.020% 99.990% ( 6600, 9900 ] 8 0.008% 99.997% ( 9900, 14000 ] 1 0.001% 99.998% ( 14000, 22000 ] 1 0.001% 99.999% ( 22000, 33000 ] 1 0.001% 100.000% ( 33000, 50000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 294111 Average: 12.6552 StdDev: 186.84 Min: 1 Median: 1.9199 Max: 35035 Percentiles: P50: 1.92 P75: 2.92 P99: 11.37 P99.9: 2027.73 P99.99: 3419.33 ------------------------------------------------------ [ 0, 1 ] 9297 3.161% 3.161% # ( 1, 2 ] 149758 50.919% 54.080% ########## ( 2, 3 ] 66700 22.679% 76.758% ##### ( 3, 4 ] 28690 9.755% 86.513% ## ( 4, 6 ] 28197 9.587% 96.100% ## ( 6, 10 ] 8144 2.769% 98.869% # ( 10, 15 ] 1400 0.476% 99.345% ( 15, 22 ] 211 0.072% 99.417% ( 22, 34 ] 34 0.012% 99.429% ( 34, 51 ] 25 0.009% 99.437% ( 51, 76 ] 17 0.006% 99.443% ( 76, 110 ] 16 0.005% 99.449% ( 110, 170 ] 45 0.015% 99.464% ( 170, 250 ] 63 0.021% 99.485% ( 250, 380 ] 125 0.043% 99.528% ( 380, 580 ] 199 0.068% 99.595% ( 580, 870 ] 270 0.092% 99.687% ( 870, 1300 ] 325 0.111% 99.798% ( 1300, 1900 ] 271 0.092% 99.890% ( 1900, 2900 ] 234 0.080% 99.969% ( 2900, 4400 ] 175 0.060% 100.029% ( 4400, 6600 ] 62 0.021% 100.050% ( 6600, 9900 ] 13 0.004% 100.054% ( 9900, 14000 ] 2 0.001% 100.055% ( 14000, 22000 ] 6 0.002% 100.057% ( 33000, 50000 ] 1 0.000% 100.057% ** Level 3 read latency histogram (micros): Count: 963803 Average: 10.7772 StdDev: 180.08 Min: 1 Median: 1.9466 Max: 48102 Percentiles: P50: 1.95 P75: 2.94 P99: 9.68 P99.9: 892.04 P99.99: 1297.02 ------------------------------------------------------ [ 0, 1 ] 17618 1.828% 1.828% ( 1, 2 ] 490453 50.887% 52.715% ########## ( 2, 3 ] 227453 23.600% 76.315% ##### ( 3, 4 ] 99568 10.331% 86.646% ## ( 4, 6 ] 97523 10.119% 96.764% ## ( 6, 10 ] 23405 2.428% 99.192% ( 10, 15 ] 3554 0.369% 99.561% ( 15, 22 ] 475 0.049% 99.611% ( 22, 34 ] 57 0.006% 99.616% ( 34, 51 ] 59 0.006% 99.623% ( 51, 76 ] 85 0.009% 99.631% ( 76, 110 ] 64 0.007% 99.638% ( 110, 170 ] 239 0.025% 99.663% ( 170, 250 ] 248 0.026% 99.689% ( 250, 380 ] 344 0.036% 99.724% ( 380, 580 ] 673 0.070% 99.794% ( 580, 870 ] 974 0.101% 99.895% ( 870, 1300 ] 921 0.096% 99.991% ( 1300, 1900 ] 572 0.059% 100.050% ( 1900, 2900 ] 454 0.047% 100.097% ( 2900, 4400 ] 522 0.054% 100.151% ( 4400, 6600 ] 167 0.017% 100.169% ( 6600, 9900 ] 33 0.003% 100.172% ( 9900, 14000 ] 5 0.001% 100.173% ( 14000, 22000 ] 8 0.001% 100.173% ( 22000, 33000 ] 6 0.001% 100.174% ( 33000, 50000 ] 3 0.000% 100.174% ** Level 4 read latency histogram (micros): Count: 745454 Average: 19.0499 StdDev: 205.27 Min: 1 Median: 1.9038 Max: 48929 Percentiles: P50: 1.90 P75: 2.89 P99: 306.34 P99.9: 1778.25 P99.99: 2582.22 ------------------------------------------------------ [ 0, 1 ] 17211 2.309% 2.309% ( 1, 2 ] 393371 52.769% 55.078% ########### ( 2, 3 ] 167056 22.410% 77.488% #### ( 3, 4 ] 70927 9.515% 87.003% ## ( 4, 6 ] 64761 8.687% 95.690% ## ( 6, 10 ] 17403 2.335% 98.025% ( 10, 15 ] 2502 0.336% 98.360% ( 15, 22 ] 313 0.042% 98.402% ( 22, 34 ] 25 0.003% 98.406% ( 34, 51 ] 31 0.004% 98.410% ( 51, 76 ] 54 0.007% 98.417% ( 76, 110 ] 590 0.079% 98.496% ( 110, 170 ] 1950 0.262% 98.758% ( 170, 250 ] 1304 0.175% 98.933% ( 250, 380 ] 1157 0.155% 99.088% ( 380, 580 ] 1699 0.228% 99.316% ( 580, 870 ] 1832 0.246% 99.562% ( 870, 1300 ] 1688 0.226% 99.788% ( 1300, 1900 ] 1047 0.140% 99.928% ( 1900, 2900 ] 672 0.090% 100.019% ( 2900, 4400 ] 712 0.096% 100.114% ( 4400, 6600 ] 197 0.026% 100.141% ( 6600, 9900 ] 43 0.006% 100.146% ( 9900, 14000 ] 6 0.001% 100.147% ( 14000, 22000 ] 2 0.000% 100.147% ( 22000, 33000 ] 2 0.000% 100.148% ( 33000, 50000 ] 1 0.000% 100.148% ** Level 5 read latency histogram (micros): Count: 51909 Average: 709.3671 StdDev: 1218.75 Min: 75 Median: 313.4063 Max: 58429 Percentiles: P50: 313.41 P75: 771.49 P99: 5588.31 P99.9: 9156.17 P99.99: 31950.05 ------------------------------------------------------ ( 51, 76 ] 1 0.002% 0.002% ( 76, 110 ] 3586 6.908% 6.910% # ( 110, 170 ] 12035 23.185% 30.095% ##### ( 170, 250 ] 7647 14.732% 44.827% ### ( 250, 380 ] 5506 10.607% 55.434% ## ( 380, 580 ] 6395 12.320% 67.753% ## ( 580, 870 ] 5697 10.975% 78.728% ## ( 870, 1300 ] 4254 8.195% 86.923% ## ( 1300, 1900 ] 2305 4.440% 91.364% # ( 1900, 2900 ] 1757 3.385% 94.749% # ( 2900, 4400 ] 1791 3.450% 98.199% # ( 4400, 6600 ] 770 1.483% 99.682% ( 6600, 9900 ] 146 0.281% 99.963% ( 9900, 14000 ] 6 0.012% 99.975% ( 14000, 22000 ] 6 0.012% 99.987% ( 22000, 33000 ] 2 0.004% 99.990% ( 33000, 50000 ] 7 0.013% 100.004% ( 50000, 75000 ] 2 0.004% 100.008% ** DB Stats ** Uptime(secs): 13.3 total, 5.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 147.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.482 H:M:S, 41.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 191.11 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.493 H:M:S, 29.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/07-09:47:01 ... thread 0: (1000000,3000000) ops and (288881.4,180355.2) ops/second in (3.461628,16.633846) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/4 2.77 GB 258.0 2.3 0.0 2.3 5.1 2.8 0.0 1.8 87.1 190.2 27.47 17.69 47 0.584 2430K 2794 0.0 0.0 L1 15/0 1.18 GB 121.0 1.2 0.8 0.4 1.2 0.8 0.0 1.5 93.5 93.0 13.01 6.42 1 13.013 1235K 5936 0.0 0.0 L3 12/0 997.25 MB 1.0 2.3 1.4 0.9 2.2 1.4 0.3 1.6 95.7 94.7 24.26 11.01 8 3.032 1046K 0 0.0 0.0 L4 178/0 9.49 GB 1.0 6.8 4.0 2.8 6.7 3.9 0.0 1.7 114.4 112.8 61.03 32.87 40 1.526 3344K 12K 0.0 0.0 L5 1132/0 93.07 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1352/4 107.48 GB 0.0 12.6 6.1 6.5 15.3 8.8 4.4 5.5 102.7 124.2 125.77 67.98 96 1.310 8056K 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.6 6.1 6.5 15.3 8.8 4.4 5.5 102.7 124.2 125.77 67.98 96 1.310 8056K 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.6 6.1 6.5 12.5 6.0 0.0 0.0 116.9 115.8 110.43 59.80 57 1.937 8056K 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 184.8 15.34 8.19 39 0.393 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.8 total, 16.6 interval Flush(GB): cumulative 2.768, interval 2.768 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.25 GB write, 929.15 MB/s write, 12.61 GB read, 768.12 MB/s read, 125.8 seconds Interval compaction: 15.25 GB write, 938.90 MB/s write, 12.61 GB read, 776.18 MB/s read, 125.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 42, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 884554 Average: 11.7339 StdDev: 136.11 Min: 1 Median: 1.6167 Max: 47247 Percentiles: P50: 1.62 P75: 2.24 P99: 216.97 P99.9: 1346.23 P99.99: 2861.45 ------------------------------------------------------ [ 0, 1 ] 135756 15.347% 15.347% ### ( 1, 2 ] 497075 56.195% 71.542% ########### ( 2, 3 ] 128621 14.541% 86.083% ### ( 3, 4 ] 49551 5.602% 91.685% # ( 4, 6 ] 44185 4.995% 96.680% # ( 6, 10 ] 10544 1.192% 97.872% ( 10, 15 ] 1777 0.201% 98.073% ( 15, 22 ] 236 0.027% 98.100% ( 22, 34 ] 24 0.003% 98.102% ( 34, 51 ] 13 0.001% 98.104% ( 51, 76 ] 25 0.003% 98.107% ( 76, 110 ] 1252 0.142% 98.248% ( 110, 170 ] 4799 0.543% 98.791% ( 170, 250 ] 3155 0.357% 99.147% ( 250, 380 ] 1999 0.226% 99.373% ( 380, 580 ] 2035 0.230% 99.604% ( 580, 870 ] 1577 0.178% 99.782% ( 870, 1300 ] 1009 0.114% 99.896% ( 1300, 1900 ] 499 0.056% 99.952% ( 1900, 2900 ] 349 0.039% 99.992% ( 2900, 4400 ] 258 0.029% 100.021% ( 4400, 6600 ] 67 0.008% 100.028% ( 6600, 9900 ] 14 0.002% 100.030% ( 9900, 14000 ] 1 0.000% 100.030% ( 14000, 22000 ] 6 0.001% 100.031% ( 22000, 33000 ] 1 0.000% 100.031% ( 33000, 50000 ] 1 0.000% 100.031% ** Level 1 read latency histogram (micros): Count: 106586 Average: 14.2838 StdDev: 206.56 Min: 1 Median: 2.5764 Max: 33950 Percentiles: P50: 2.58 P75: 3.89 P99: 166.18 P99.9: 2548.28 P99.99: 6740.83 ------------------------------------------------------ [ 0, 1 ] 867 0.813% 0.813% ( 1, 2 ] 34446 32.318% 33.131% ###### ( 2, 3 ] 31191 29.264% 62.395% ###### ( 3, 4 ] 15053 14.123% 76.518% ### ( 4, 6 ] 17130 16.072% 92.589% ### ( 6, 10 ] 4796 4.500% 97.089% # ( 10, 15 ] 863 0.810% 97.898% ( 15, 22 ] 269 0.252% 98.151% ( 22, 34 ] 20 0.019% 98.170% ( 34, 51 ] 4 0.004% 98.173% ( 51, 76 ] 18 0.017% 98.190% ( 76, 110 ] 248 0.233% 98.423% ( 110, 170 ] 657 0.616% 99.039% ( 170, 250 ] 296 0.278% 99.317% ( 250, 380 ] 151 0.142% 99.459% ( 380, 580 ] 162 0.152% 99.611% ( 580, 870 ] 130 0.122% 99.733% ( 870, 1300 ] 83 0.078% 99.810% ( 1300, 1900 ] 63 0.059% 99.870% ( 1900, 2900 ] 50 0.047% 99.916% ( 2900, 4400 ] 57 0.053% 99.970% ( 4400, 6600 ] 21 0.020% 99.990% ( 6600, 9900 ] 8 0.008% 99.997% ( 9900, 14000 ] 1 0.001% 99.998% ( 14000, 22000 ] 1 0.001% 99.999% ( 22000, 33000 ] 1 0.001% 100.000% ( 33000, 50000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 294111 Average: 12.6552 StdDev: 186.84 Min: 1 Median: 1.9199 Max: 35035 Percentiles: P50: 1.92 P75: 2.92 P99: 11.37 P99.9: 2027.73 P99.99: 3419.33 ------------------------------------------------------ [ 0, 1 ] 9297 3.161% 3.161% # ( 1, 2 ] 149758 50.919% 54.080% ########## ( 2, 3 ] 66700 22.679% 76.758% ##### ( 3, 4 ] 28690 9.755% 86.513% ## ( 4, 6 ] 28197 9.587% 96.100% ## ( 6, 10 ] 8144 2.769% 98.869% # ( 10, 15 ] 1400 0.476% 99.345% ( 15, 22 ] 211 0.072% 99.417% ( 22, 34 ] 34 0.012% 99.429% ( 34, 51 ] 25 0.009% 99.437% ( 51, 76 ] 17 0.006% 99.443% ( 76, 110 ] 16 0.005% 99.449% ( 110, 170 ] 45 0.015% 99.464% ( 170, 250 ] 63 0.021% 99.485% ( 250, 380 ] 125 0.043% 99.528% ( 380, 580 ] 199 0.068% 99.595% ( 580, 870 ] 270 0.092% 99.687% ( 870, 1300 ] 325 0.111% 99.798% ( 1300, 1900 ] 271 0.092% 99.890% ( 1900, 2900 ] 234 0.080% 99.969% ( 2900, 4400 ] 175 0.060% 100.029% ( 4400, 6600 ] 62 0.021% 100.050% ( 6600, 9900 ] 13 0.004% 100.054% ( 9900, 14000 ] 2 0.001% 100.055% ( 14000, 22000 ] 6 0.002% 100.057% ( 33000, 50000 ] 1 0.000% 100.057% ** Level 3 read latency histogram (micros): Count: 964509 Average: 10.9081 StdDev: 180.11 Min: 1 Median: 1.9474 Max: 48102 Percentiles: P50: 1.95 P75: 2.95 P99: 9.80 P99.9: 891.71 P99.99: 1296.99 ------------------------------------------------------ [ 0, 1 ] 17618 1.827% 1.827% ( 1, 2 ] 490453 50.850% 52.677% ########## ( 2, 3 ] 227453 23.582% 76.259% ##### ( 3, 4 ] 99568 10.323% 86.582% ## ( 4, 6 ] 97523 10.111% 96.693% ## ( 6, 10 ] 23405 2.427% 99.120% ( 10, 15 ] 3554 0.368% 99.488% ( 15, 22 ] 475 0.049% 99.538% ( 22, 34 ] 57 0.006% 99.543% ( 34, 51 ] 59 0.006% 99.550% ( 51, 76 ] 85 0.009% 99.558% ( 76, 110 ] 194 0.020% 99.579% ( 110, 170 ] 583 0.060% 99.639% ( 170, 250 ] 363 0.038% 99.677% ( 250, 380 ] 395 0.041% 99.718% ( 380, 580 ] 723 0.075% 99.793% ( 580, 870 ] 990 0.103% 99.895% ( 870, 1300 ] 921 0.095% 99.991% ( 1300, 1900 ] 572 0.059% 100.050% ( 1900, 2900 ] 454 0.047% 100.097% ( 2900, 4400 ] 522 0.054% 100.151% ( 4400, 6600 ] 167 0.017% 100.168% ( 6600, 9900 ] 33 0.003% 100.172% ( 9900, 14000 ] 5 0.001% 100.172% ( 14000, 22000 ] 8 0.001% 100.173% ( 22000, 33000 ] 6 0.001% 100.174% ( 33000, 50000 ] 3 0.000% 100.174% ** Level 4 read latency histogram (micros): Count: 753333 Average: 20.6392 StdDev: 205.35 Min: 1 Median: 1.9135 Max: 48929 Percentiles: P50: 1.91 P75: 2.92 P99: 371.29 P99.9: 1776.05 P99.99: 2583.48 ------------------------------------------------------ [ 0, 1 ] 17211 2.285% 2.285% ( 1, 2 ] 393474 52.231% 54.516% ########## ( 2, 3 ] 167308 22.209% 76.725% #### ( 3, 4 ] 71169 9.447% 86.172% ## ( 4, 6 ] 65033 8.633% 94.805% ## ( 6, 10 ] 17432 2.314% 97.119% ( 10, 15 ] 2508 0.333% 97.452% ( 15, 22 ] 315 0.042% 97.493% ( 22, 34 ] 25 0.003% 97.497% ( 34, 51 ] 31 0.004% 97.501% ( 51, 76 ] 57 0.008% 97.508% ( 76, 110 ] 1875 0.249% 97.757% ( 110, 170 ] 5275 0.700% 98.458% ( 170, 250 ] 2550 0.338% 98.796% ( 250, 380 ] 1647 0.219% 99.015% ( 380, 580 ] 2120 0.281% 99.296% ( 580, 870 ] 1997 0.265% 99.561% ( 870, 1300 ] 1718 0.228% 99.789% ( 1300, 1900 ] 1052 0.140% 99.929% ( 1900, 2900 ] 674 0.089% 100.018% ( 2900, 4400 ] 713 0.095% 100.113% ( 4400, 6600 ] 197 0.026% 100.139% ( 6600, 9900 ] 43 0.006% 100.145% ( 9900, 14000 ] 6 0.001% 100.146% ( 14000, 22000 ] 2 0.000% 100.146% ( 22000, 33000 ] 2 0.000% 100.146% ( 33000, 50000 ] 1 0.000% 100.146% ** Level 5 read latency histogram (micros): Count: 98661 Average: 464.8355 StdDev: 926.36 Min: 75 Median: 183.7421 Max: 58429 Percentiles: P50: 183.74 P75: 437.40 P99: 4353.50 P99.9: 8009.03 P99.99: 13408.16 ------------------------------------------------------ ( 51, 76 ] 9 0.009% 0.009% ( 76, 110 ] 12084 12.248% 12.257% ## ( 110, 170 ] 34548 35.017% 47.274% ####### ( 170, 250 ] 15657 15.869% 63.143% ### ( 250, 380 ] 9009 9.131% 72.275% ## ( 380, 580 ] 9369 9.496% 81.771% ## ( 580, 870 ] 6704 6.795% 88.566% # ( 870, 1300 ] 4432 4.492% 93.058% # ( 1300, 1900 ] 2336 2.368% 95.426% ( 1900, 2900 ] 1788 1.812% 97.238% ( 2900, 4400 ] 1794 1.818% 99.056% ( 4400, 6600 ] 770 0.780% 99.837% ( 6600, 9900 ] 146 0.148% 99.985% ( 9900, 14000 ] 6 0.006% 99.991% ( 14000, 22000 ] 6 0.006% 99.997% ( 22000, 33000 ] 2 0.002% 99.999% ( 33000, 50000 ] 7 0.007% 100.006% ( 50000, 75000 ] 2 0.002% 100.008% ** DB Stats ** Uptime(secs): 16.8 total, 3.5 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 175.64 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.482 H:M:S, 32.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 284.31 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/07-09:47:04 ... thread 0: (1000000,4000000) ops and (292477.9,199472.3) ops/second in (3.419062,20.052908) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/4 3.75 GB 358.7 3.3 0.0 3.3 7.1 3.7 0.0 1.9 98.9 210.6 34.38 24.13 67 0.513 3454K 3801 0.0 0.0 L1 15/0 1.18 GB 121.0 1.2 0.8 0.4 1.2 0.8 0.0 1.5 93.5 93.0 13.01 6.42 1 13.013 1235K 5936 0.0 0.0 L3 12/0 997.25 MB 1.0 2.3 1.4 0.9 2.2 1.4 0.3 1.6 95.7 94.7 24.26 11.01 8 3.032 1046K 0 0.0 0.0 L4 178/0 9.49 GB 1.0 6.8 4.0 2.8 6.7 3.9 0.0 1.7 114.4 112.8 61.03 32.87 40 1.526 3344K 12K 0.0 0.0 L5 1132/0 93.07 GB 1.0 0.0 0.0 0.0 0.0 0.0 4.1 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1356/4 108.46 GB 0.0 13.6 6.1 7.5 17.2 9.8 4.4 4.6 104.9 132.9 132.69 74.43 116 1.144 9079K 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.6 6.1 7.5 17.2 9.8 4.4 4.6 104.9 132.9 132.69 74.43 116 1.144 9079K 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.6 6.1 7.5 13.5 6.0 0.0 0.0 121.8 120.7 114.27 63.56 61 1.873 9079K 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.8 3.8 0.0 0.0 0.0 208.7 18.41 10.87 55 0.335 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.2 total, 20.1 interval Flush(GB): cumulative 3.753, interval 3.753 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.22 GB write, 871.71 MB/s write, 13.59 GB read, 688.12 MB/s read, 132.7 seconds Interval compaction: 17.22 GB write, 879.30 MB/s write, 13.59 GB read, 694.12 MB/s read, 132.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 42, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1147821 Average: 10.0652 StdDev: 120.43 Min: 0 Median: 1.5507 Max: 47247 Percentiles: P50: 1.55 P75: 1.99 P99: 181.27 P99.9: 1177.22 P99.99: 2569.17 ------------------------------------------------------ [ 0, 1 ] 217087 18.913% 18.913% #### ( 1, 2 ] 647969 56.452% 75.365% ########### ( 2, 3 ] 145957 12.716% 88.081% ### ( 3, 4 ] 55680 4.851% 92.932% # ( 4, 6 ] 47813 4.166% 97.098% # ( 6, 10 ] 11174 0.973% 98.071% ( 10, 15 ] 1874 0.163% 98.234% ( 15, 22 ] 249 0.022% 98.256% ( 22, 34 ] 26 0.002% 98.258% ( 34, 51 ] 14 0.001% 98.259% ( 51, 76 ] 28 0.002% 98.262% ( 76, 110 ] 1613 0.141% 98.402% ( 110, 170 ] 6302 0.549% 98.951% ( 170, 250 ] 3966 0.346% 99.297% ( 250, 380 ] 2296 0.200% 99.497% ( 380, 580 ] 2251 0.196% 99.693% ( 580, 870 ] 1646 0.143% 99.837% ( 870, 1300 ] 1022 0.089% 99.926% ( 1300, 1900 ] 505 0.044% 99.970% ( 1900, 2900 ] 353 0.031% 100.000% ( 2900, 4400 ] 260 0.023% 100.023% ( 4400, 6600 ] 67 0.006% 100.029% ( 6600, 9900 ] 14 0.001% 100.030% ( 9900, 14000 ] 1 0.000% 100.030% ( 14000, 22000 ] 6 0.001% 100.031% ( 22000, 33000 ] 1 0.000% 100.031% ( 33000, 50000 ] 1 0.000% 100.031% ** Level 1 read latency histogram (micros): Count: 107773 Average: 16.0194 StdDev: 206.52 Min: 1 Median: 2.5947 Max: 33950 Percentiles: P50: 2.59 P75: 3.95 P99: 218.12 P99.9: 2531.90 P99.99: 6691.86 ------------------------------------------------------ [ 0, 1 ] 867 0.804% 0.804% ( 1, 2 ] 34454 31.969% 32.774% ###### ( 2, 3 ] 31216 28.965% 61.738% ###### ( 3, 4 ] 15065 13.978% 75.717% ### ( 4, 6 ] 17148 15.911% 91.628% ### ( 6, 10 ] 4800 4.454% 96.082% # ( 10, 15 ] 864 0.802% 96.883% ( 15, 22 ] 269 0.250% 97.133% ( 22, 34 ] 20 0.019% 97.151% ( 34, 51 ] 4 0.004% 97.155% ( 51, 76 ] 18 0.017% 97.172% ( 76, 110 ] 443 0.411% 97.583% ( 110, 170 ] 1247 1.157% 98.740% ( 170, 250 ] 466 0.432% 99.172% ( 250, 380 ] 237 0.220% 99.392% ( 380, 580 ] 222 0.206% 99.598% ( 580, 870 ] 146 0.135% 99.734% ( 870, 1300 ] 84 0.078% 99.812% ( 1300, 1900 ] 63 0.058% 99.870% ( 1900, 2900 ] 51 0.047% 99.917% ( 2900, 4400 ] 57 0.053% 99.970% ( 4400, 6600 ] 21 0.019% 99.990% ( 6600, 9900 ] 8 0.007% 99.997% ( 9900, 14000 ] 1 0.001% 99.998% ( 14000, 22000 ] 1 0.001% 99.999% ( 22000, 33000 ] 1 0.001% 100.000% ( 33000, 50000 ] 1 0.001% 100.001% ** Level 2 read latency histogram (micros): Count: 294111 Average: 12.6552 StdDev: 186.84 Min: 1 Median: 1.9199 Max: 35035 Percentiles: P50: 1.92 P75: 2.92 P99: 11.37 P99.9: 2027.73 P99.99: 3419.33 ------------------------------------------------------ [ 0, 1 ] 9297 3.161% 3.161% # ( 1, 2 ] 149758 50.919% 54.080% ########## ( 2, 3 ] 66700 22.679% 76.758% ##### ( 3, 4 ] 28690 9.755% 86.513% ## ( 4, 6 ] 28197 9.587% 96.100% ## ( 6, 10 ] 8144 2.769% 98.869% # ( 10, 15 ] 1400 0.476% 99.345% ( 15, 22 ] 211 0.072% 99.417% ( 22, 34 ] 34 0.012% 99.429% ( 34, 51 ] 25 0.009% 99.437% ( 51, 76 ] 17 0.006% 99.443% ( 76, 110 ] 16 0.005% 99.449% ( 110, 170 ] 45 0.015% 99.464% ( 170, 250 ] 63 0.021% 99.485% ( 250, 380 ] 125 0.043% 99.528% ( 380, 580 ] 199 0.068% 99.595% ( 580, 870 ] 270 0.092% 99.687% ( 870, 1300 ] 325 0.111% 99.798% ( 1300, 1900 ] 271 0.092% 99.890% ( 1900, 2900 ] 234 0.080% 99.969% ( 2900, 4400 ] 175 0.060% 100.029% ( 4400, 6600 ] 62 0.021% 100.050% ( 6600, 9900 ] 13 0.004% 100.054% ( 9900, 14000 ] 2 0.001% 100.055% ( 14000, 22000 ] 6 0.002% 100.057% ( 33000, 50000 ] 1 0.000% 100.057% ** Level 3 read latency histogram (micros): Count: 965200 Average: 11.0347 StdDev: 180.14 Min: 1 Median: 1.9481 Max: 48102 Percentiles: P50: 1.95 P75: 2.95 P99: 9.92 P99.9: 891.83 P99.99: 1296.96 ------------------------------------------------------ [ 0, 1 ] 17618 1.825% 1.825% ( 1, 2 ] 490453 50.814% 52.639% ########## ( 2, 3 ] 227453 23.565% 76.204% ##### ( 3, 4 ] 99568 10.316% 86.520% ## ( 4, 6 ] 97523 10.104% 96.624% ## ( 6, 10 ] 23405 2.425% 99.049% ( 10, 15 ] 3554 0.368% 99.417% ( 15, 22 ] 475 0.049% 99.466% ( 22, 34 ] 57 0.006% 99.472% ( 34, 51 ] 59 0.006% 99.478% ( 51, 76 ] 85 0.009% 99.487% ( 76, 110 ] 338 0.035% 99.522% ( 110, 170 ] 900 0.093% 99.615% ( 170, 250 ] 480 0.050% 99.665% ( 250, 380 ] 447 0.046% 99.711% ( 380, 580 ] 769 0.080% 99.791% ( 580, 870 ] 1004 0.104% 99.895% ( 870, 1300 ] 922 0.096% 99.991% ( 1300, 1900 ] 572 0.059% 100.050% ( 1900, 2900 ] 454 0.047% 100.097% ( 2900, 4400 ] 522 0.054% 100.151% ( 4400, 6600 ] 167 0.017% 100.168% ( 6600, 9900 ] 33 0.003% 100.172% ( 9900, 14000 ] 5 0.001% 100.172% ( 14000, 22000 ] 8 0.001% 100.173% ( 22000, 33000 ] 6 0.001% 100.174% ( 33000, 50000 ] 3 0.000% 100.174% ** Level 4 read latency histogram (micros): Count: 761332 Average: 22.2702 StdDev: 205.34 Min: 1 Median: 1.9236 Max: 48929 Percentiles: P50: 1.92 P75: 2.96 P99: 411.39 P99.9: 1774.24 P99.99: 2584.17 ------------------------------------------------------ [ 0, 1 ] 17211 2.261% 2.261% ( 1, 2 ] 393522 51.689% 53.949% ########## ( 2, 3 ] 167423 21.991% 75.940% #### ( 3, 4 ] 71235 9.357% 85.297% ## ( 4, 6 ] 65132 8.555% 93.852% ## ( 6, 10 ] 17442 2.291% 96.143% ( 10, 15 ] 2515 0.330% 96.473% ( 15, 22 ] 316 0.042% 96.515% ( 22, 34 ] 25 0.003% 96.518% ( 34, 51 ] 31 0.004% 96.522% ( 51, 76 ] 59 0.008% 96.530% ( 76, 110 ] 3411 0.448% 96.978% ( 110, 170 ] 8995 1.181% 98.159% ( 170, 250 ] 3829 0.503% 98.662% ( 250, 380 ] 2167 0.285% 98.947% ( 380, 580 ] 2585 0.340% 99.286% ( 580, 870 ] 2107 0.277% 99.563% ( 870, 1300 ] 1731 0.227% 99.790% ( 1300, 1900 ] 1056 0.139% 99.929% ( 1900, 2900 ] 678 0.089% 100.018% ( 2900, 4400 ] 713 0.094% 100.112% ( 4400, 6600 ] 197 0.026% 100.138% ( 6600, 9900 ] 43 0.006% 100.143% ( 9900, 14000 ] 6 0.001% 100.144% ( 14000, 22000 ] 2 0.000% 100.144% ( 22000, 33000 ] 2 0.000% 100.145% ( 33000, 50000 ] 1 0.000% 100.145% ** Level 5 read latency histogram (micros): Count: 145820 Average: 374.4462 StdDev: 776.96 Min: 75 Median: 163.6054 Max: 58429 Percentiles: P50: 163.61 P75: 323.74 P99: 3957.18 P99.9: 6875.30 P99.99: 9841.64 ------------------------------------------------------ ( 51, 76 ] 14 0.010% 0.010% ( 76, 110 ] 21400 14.676% 14.685% ### ( 110, 170 ] 57639 39.527% 54.213% ######## ( 170, 250 ] 23340 16.006% 70.219% ### ( 250, 380 ] 12292 8.430% 78.648% ## ( 380, 580 ] 12209 8.373% 87.021% ## ( 580, 870 ] 7510 5.150% 92.171% # ( 870, 1300 ] 4525 3.103% 95.274% # ( 1300, 1900 ] 2353 1.614% 96.888% ( 1900, 2900 ] 1814 1.244% 98.132% ( 2900, 4400 ] 1796 1.232% 99.364% ( 4400, 6600 ] 770 0.528% 99.892% ( 6600, 9900 ] 146 0.100% 99.992% ( 9900, 14000 ] 6 0.004% 99.996% ( 14000, 22000 ] 6 0.004% 100.000% ( 22000, 33000 ] 2 0.001% 100.001% ( 33000, 50000 ] 7 0.005% 100.006% ( 50000, 75000 ] 2 0.001% 100.008% ** DB Stats ** Uptime(secs): 20.2 total, 3.4 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 194.61 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.482 H:M:S, 27.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 287.86 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 237.641 micros/op 16808 ops/sec 60.147 seconds 1010996 operations; 16.3 MB/s (252999 of 252999 found) Microseconds per read: Count: 1010996 Average: 237.6434 StdDev: 704.96 Min: 2 Median: 177.0000 Max: 87934 Percentiles: P50: 177.00 P75: 231.89 P99: 1078.72 P99.9: 10947.03 P99.99: 29918.66 ------------------------------------------------------ ( 1, 2 ] 4 0.000% 0.000% ( 2, 3 ] 29 0.003% 0.003% ( 3, 4 ] 57 0.006% 0.009% ( 4, 6 ] 106 0.010% 0.019% ( 6, 10 ] 237 0.023% 0.043% ( 10, 15 ] 2072 0.205% 0.248% ( 15, 22 ] 4353 0.431% 0.678% ( 22, 34 ] 12348 1.221% 1.900% ( 34, 51 ] 3337 0.330% 2.230% ( 51, 76 ] 464 0.046% 2.276% ( 76, 110 ] 13966 1.381% 3.657% ( 110, 170 ] 436294 43.155% 46.812% ######### ( 170, 250 ] 368352 36.435% 83.247% ####### ( 250, 380 ] 104038 10.291% 93.537% ## ( 380, 580 ] 38217 3.780% 97.317% # ( 580, 870 ] 14154 1.400% 98.717% ( 870, 1300 ] 5888 0.582% 99.300% ( 1300, 1900 ] 2470 0.244% 99.544% ( 1900, 2900 ] 1599 0.158% 99.702% ( 2900, 4400 ] 1209 0.120% 99.822% ( 4400, 6600 ] 425 0.042% 99.864% ( 6600, 9900 ] 247 0.024% 99.888% ( 9900, 14000 ] 466 0.046% 99.934% ( 14000, 22000 ] 447 0.044% 99.979% ( 22000, 33000 ] 161 0.016% 99.994% ( 33000, 50000 ] 40 0.004% 99.998% ( 50000, 75000 ] 10 0.001% 99.999% ( 75000, 110000 ] 6 0.001% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 21249347 rocksdb.bloom.filter.full.positive COUNT : 1217774 rocksdb.bloom.filter.full.true.positive COUNT : 1010539 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 457 rocksdb.memtable.miss COUNT : 1010539 rocksdb.l0.hit COUNT : 42632 rocksdb.l1.hit COUNT : 11506 rocksdb.l2andup.hit COUNT : 956401 rocksdb.compaction.key.drop.new COUNT : 175285 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4847075 rocksdb.number.keys.read COUNT : 1010996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 5002181400 rocksdb.bytes.read COUNT : 1010996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1579 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 41848283 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 17660662102 rocksdb.compact.write.bytes COUNT : 17480610516 rocksdb.flush.write.bytes COUNT : 4954402105 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 26 rocksdb.number.superversion_acquires COUNT : 896 rocksdb.number.superversion_releases COUNT : 866 rocksdb.number.superversion_cleanups COUNT : 225 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 213 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4914329600 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1798144 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21901763434 rocksdb.non.last.level.read.count COUNT : 5149512 rocksdb.block.checksum.compute.count COUNT : 5509508 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 175.439062 P95 : 456.012234 P99 : 1077.720374 P100 : 87932.000000 COUNT : 1010996 SUM : 239304484 rocksdb.db.write.micros P50 : 1.532911 P95 : 4.168522 P99 : 9.034896 P100 : 1035608.000000 COUNT : 4847075 SUM : 53813506 rocksdb.compaction.times.micros P50 : 1334615.384615 P95 : 3512500.000000 P99 : 11270000.000000 P100 : 13012939.000000 COUNT : 65 SUM : 118104009 rocksdb.compaction.times.cpu_micros P50 : 945000.000000 P95 : 1812500.000000 P99 : 5070000.000000 P100 : 6422780.000000 COUNT : 65 SUM : 67327577 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 753.292683 P95 : 148250.000000 P99 : 581160.000000 P100 : 581160.000000 COUNT : 69 SUM : 1481116 rocksdb.compaction.outfile.sync.micros P50 : 57031.250000 P95 : 364291.666667 P99 : 701950.000000 P100 : 910529.000000 COUNT : 209 SUM : 20379541 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 198.000000 P95 : 4345.000000 P99 : 6413.000000 P100 : 9630.000000 COUNT : 151 SUM : 180343 rocksdb.table.open.io.micros P50 : 9001.626898 P95 : 20504.433498 P99 : 29923.142857 P100 : 84186.000000 COUNT : 1579 SUM : 14939480 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.934257 P95 : 77.493449 P99 : 169.766467 P100 : 48933.000000 COUNT : 4277306 SUM : 72946963 rocksdb.read.block.get.micros P50 : 135.816807 P95 : 312.722000 P99 : 951.946080 P100 : 58501.000000 COUNT : 1230634 SUM : 198945279 rocksdb.write.raw.block.micros P50 : 0.533925 P95 : 1.311994 P99 : 3.532914 P100 : 53765.000000 COUNT : 5433949 SUM : 16188395 rocksdb.numfiles.in.singlecompaction P50 : 3.029412 P95 : 5.300000 P99 : 24.160000 P100 : 32.000000 COUNT : 82 SUM : 311 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.591841 P95 : 2318.463546 P99 : 2784.410374 P100 : 1035451.000000 COUNT : 142836 SUM : 41860202 rocksdb.sst.read.micros P50 : 2.318710 P95 : 167.388631 P99 : 477.232862 P100 : 58429.000000 COUNT : 5149569 SUM : 255202332 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1010996 SUM : 1010996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4847075 SUM : 5002181400 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219285.714286 P95 : 1035666.666667 P99 : 1353627.000000 P100 : 1353627.000000 COUNT : 69 SUM : 21226713 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 226 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0