[2024-12-05 20:34:05.175990] Starting SPDK v25.01-pre git sha1 98eca6fa0 / DPDK 24.03.0 initialization... [2024-12-05 20:34:05.176114] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2417757 ] Set seed to 1733427245173423 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-05 20:34:05.348683] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-05 20:34:05.490530] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-05 20:34:05.490593] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2417757' to capture a snapshot of events at runtime. [2024-12-05 20:34:05.490612] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-05 20:34:05.490631] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-05 20:34:05.490645] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2417757 for offline analysis/debug. [2024-12-05 20:34:05.492292] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Thu Dec 5 20:34:12 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/12/05-20:34:18 ... thread 0: (1000000,1000000) ops and (160398.8,160398.8) ops/second in (6.234460,6.234460) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/11 2.84 GB 56.4 0.3 0.0 0.3 1.2 0.9 0.0 1.4 42.2 160.1 7.45 4.30 12 0.621 319K 1819 0.0 0.0 L1 6/6 725.70 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 17/1 1.00 GB 0.9 3.4 1.6 1.8 3.3 1.5 0.0 2.1 137.8 135.3 25.05 17.92 13 1.927 1891K 19K 0.0 0.0 L4 128/8 10.21 GB 1.0 1.3 1.2 0.1 1.2 1.2 0.2 1.1 130.9 129.8 9.84 6.88 6 1.640 1220K 0 0.0 0.0 L5 120/0 10.15 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 286/26 24.92 GB 0.0 4.9 2.7 2.2 5.7 3.5 0.9 6.7 119.3 138.4 42.34 29.11 31 1.366 3431K 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 4.9 2.7 2.2 5.7 3.5 0.9 6.7 119.3 138.4 42.34 29.11 31 1.366 3431K 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.9 2.7 2.2 4.9 2.7 0.0 0.0 137.6 135.6 36.73 26.34 20 1.836 3431K 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 156.9 5.61 2.77 11 0.510 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.4 total, 6.2 interval Flush(GB): cumulative 0.860, interval 0.860 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.72 GB write, 918.07 MB/s write, 4.93 GB read, 791.77 MB/s read, 42.3 seconds Interval compaction: 5.72 GB write, 939.24 MB/s write, 4.93 GB read, 810.03 MB/s read, 42.3 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 14, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 14, interval: 14 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 169504 Average: 64.0660 StdDev: 397.20 Min: 1 Median: 2.4363 Max: 36621 Percentiles: P50: 2.44 P75: 3.34 P99: 1487.56 P99.9: 4249.67 P99.99: 6375.29 ------------------------------------------------------ [ 0, 1 ] 1810 1.068% 1.068% ( 1, 2 ] 54392 32.089% 33.157% ###### ( 2, 3 ] 65440 38.607% 71.763% ######## ( 3, 4 ] 16285 9.607% 81.371% ## ( 4, 6 ] 16729 9.869% 91.240% ## ( 6, 10 ] 4081 2.408% 93.648% ( 10, 15 ] 1026 0.605% 94.253% ( 15, 22 ] 83 0.049% 94.302% ( 22, 34 ] 6 0.004% 94.306% ( 34, 51 ] 4 0.002% 94.308% ( 51, 76 ] 9 0.005% 94.313% ( 76, 110 ] 3 0.002% 94.315% ( 110, 170 ] 75 0.044% 94.359% ( 170, 250 ] 400 0.236% 94.595% ( 250, 380 ] 962 0.568% 95.163% ( 380, 580 ] 1769 1.044% 96.207% ( 580, 870 ] 2540 1.498% 97.705% ( 870, 1300 ] 1928 1.137% 98.843% ( 1300, 1900 ] 854 0.504% 99.346% ( 1900, 2900 ] 512 0.302% 99.648% ( 2900, 4400 ] 474 0.280% 99.928% ( 4400, 6600 ] 117 0.069% 99.997% ( 6600, 9900 ] 19 0.011% 100.008% ( 9900, 14000 ] 4 0.002% 100.011% ( 14000, 22000 ] 5 0.003% 100.014% ( 22000, 33000 ] 3 0.002% 100.015% ( 33000, 50000 ] 3 0.002% 100.017% ** Level 1 read latency histogram (micros): Count: 681 Average: 1506.7283 StdDev: 3820.98 Min: 134 Median: 758.1429 Max: 37892 Percentiles: P50: 758.14 P75: 1192.50 P99: 27761.25 P99.9: 37892.00 P99.99: 37892.00 ------------------------------------------------------ ( 110, 170 ] 11 1.615% 1.615% ( 170, 250 ] 19 2.790% 4.405% # ( 250, 380 ] 80 11.747% 16.153% ## ( 380, 580 ] 123 18.062% 34.214% #### ( 580, 870 ] 175 25.698% 59.912% ##### ( 870, 1300 ] 137 20.117% 80.029% #### ( 1300, 1900 ] 58 8.517% 88.546% ## ( 1900, 2900 ] 23 3.377% 91.924% # ( 2900, 4400 ] 32 4.699% 96.623% # ( 4400, 6600 ] 9 1.322% 97.944% ( 6600, 9900 ] 1 0.147% 98.091% ( 14000, 22000 ] 2 0.294% 98.385% ( 22000, 33000 ] 8 1.175% 99.559% ( 33000, 50000 ] 3 0.441% 100.000% ** Level 2 read latency histogram (micros): Count: 242873 Average: 11.0256 StdDev: 194.31 Min: 1 Median: 2.4901 Max: 32903 Percentiles: P50: 2.49 P75: 3.37 P99: 12.31 P99.9: 856.98 P99.99: 1246.18 ------------------------------------------------------ [ 0, 1 ] 1328 0.547% 0.547% ( 1, 2 ] 71752 29.543% 30.090% ###### ( 2, 3 ] 98658 40.621% 70.711% ######## ( 3, 4 ] 28489 11.730% 82.441% ## ( 4, 6 ] 30734 12.654% 95.095% ### ( 6, 10 ] 8504 3.501% 98.597% # ( 10, 15 ] 2116 0.871% 99.468% ( 15, 22 ] 346 0.142% 99.610% ( 22, 34 ] 33 0.014% 99.624% ( 34, 51 ] 41 0.017% 99.641% ( 51, 76 ] 38 0.016% 99.657% ( 76, 110 ] 12 0.005% 99.662% ( 110, 170 ] 22 0.009% 99.671% ( 170, 250 ] 21 0.009% 99.679% ( 250, 380 ] 59 0.024% 99.704% ( 380, 580 ] 182 0.075% 99.778% ( 580, 870 ] 309 0.127% 99.906% ( 870, 1300 ] 234 0.096% 100.002% ( 1300, 1900 ] 136 0.056% 100.058% ( 1900, 2900 ] 113 0.047% 100.105% ( 2900, 4400 ] 103 0.042% 100.147% ( 4400, 6600 ] 23 0.009% 100.156% ( 6600, 9900 ] 10 0.004% 100.161% ( 9900, 14000 ] 2 0.001% 100.161% ( 14000, 22000 ] 7 0.003% 100.164% ( 22000, 33000 ] 3 0.001% 100.166% ** Level 3 read latency histogram (micros): Count: 624067 Average: 10.3165 StdDev: 129.66 Min: 1 Median: 2.4884 Max: 15164 Percentiles: P50: 2.49 P75: 3.31 P99: 10.30 P99.9: 458.16 P99.99: 634.19 ------------------------------------------------------ [ 0, 1 ] 2463 0.395% 0.395% ( 1, 2 ] 182897 29.307% 29.702% ###### ( 2, 3 ] 259340 41.556% 71.258% ######## ( 3, 4 ] 76096 12.194% 83.452% ## ( 4, 6 ] 77195 12.370% 95.822% ## ( 6, 10 ] 19571 3.136% 98.958% # ( 10, 15 ] 4399 0.705% 99.663% ( 15, 22 ] 504 0.081% 99.743% ( 22, 34 ] 47 0.008% 99.751% ( 34, 51 ] 80 0.013% 99.764% ( 51, 76 ] 135 0.022% 99.785% ( 76, 110 ] 24 0.004% 99.789% ( 110, 170 ] 40 0.006% 99.796% ( 170, 250 ] 101 0.016% 99.812% ( 250, 380 ] 293 0.047% 99.859% ( 380, 580 ] 660 0.106% 99.964% ( 580, 870 ] 854 0.137% 100.101% ( 870, 1300 ] 666 0.107% 100.208% ( 1300, 1900 ] 365 0.058% 100.266% ( 1900, 2900 ] 267 0.043% 100.309% ( 2900, 4400 ] 283 0.045% 100.355% ( 4400, 6600 ] 43 0.007% 100.361% ( 6600, 9900 ] 19 0.003% 100.365% ( 9900, 14000 ] 4 0.001% 100.365% ( 14000, 22000 ] 1 0.000% 100.365% ** Level 4 read latency histogram (micros): Count: 90162 Average: 104.7165 StdDev: 489.44 Min: 1 Median: 2.3940 Max: 26052 Percentiles: P50: 2.39 P75: 3.60 P99: 2261.82 P99.9: 5366.30 P99.99: 6489.08 ------------------------------------------------------ [ 0, 1 ] 1500 1.664% 1.664% ( 1, 2 ] 32362 35.893% 37.557% ####### ( 2, 3 ] 28479 31.586% 69.143% ###### ( 3, 4 ] 8799 9.759% 78.902% ## ( 4, 6 ] 7869 8.728% 87.630% ## ( 6, 10 ] 2433 2.698% 90.329% # ( 10, 15 ] 513 0.569% 90.897% ( 15, 22 ] 59 0.065% 90.963% ( 22, 34 ] 7 0.008% 90.971% ( 34, 51 ] 8 0.009% 90.980% ( 51, 76 ] 16 0.018% 90.997% ( 76, 110 ] 5 0.006% 91.003% ( 110, 170 ] 77 0.085% 91.088% ( 170, 250 ] 308 0.342% 91.430% ( 250, 380 ] 797 0.884% 92.314% ( 380, 580 ] 1488 1.650% 93.964% ( 580, 870 ] 2090 2.318% 96.282% ( 870, 1300 ] 1596 1.770% 98.052% ( 1300, 1900 ] 694 0.770% 98.822% ( 1900, 2900 ] 446 0.495% 99.317% ( 2900, 4400 ] 457 0.507% 99.824% ( 4400, 6600 ] 159 0.176% 100.000% ( 6600, 9900 ] 32 0.035% 100.035% ( 9900, 14000 ] 6 0.007% 100.042% ( 14000, 22000 ] 2 0.002% 100.044% ( 22000, 33000 ] 3 0.003% 100.048% ** Level 5 read latency histogram (micros): Count: 7340 Average: 1123.1669 StdDev: 1148.73 Min: 100 Median: 779.5964 Max: 18898 Percentiles: P50: 779.60 P75: 1234.81 P99: 6010.34 P99.9: 9633.72 P99.99: 18898.00 ------------------------------------------------------ ( 76, 110 ] 3 0.041% 0.041% ( 110, 170 ] 68 0.926% 0.967% ( 170, 250 ] 267 3.638% 4.605% # ( 250, 380 ] 706 9.619% 14.223% ## ( 380, 580 ] 1330 18.120% 32.343% #### ( 580, 870 ] 1883 25.654% 57.997% ##### ( 870, 1300 ] 1471 20.041% 78.038% #### ( 1300, 1900 ] 641 8.733% 86.771% ## ( 1900, 2900 ] 373 5.082% 91.853% # ( 2900, 4400 ] 417 5.681% 97.534% # ( 4400, 6600 ] 147 2.003% 99.537% ( 6600, 9900 ] 29 0.395% 99.932% ( 9900, 14000 ] 3 0.041% 99.973% ( 14000, 22000 ] 2 0.027% 100.000% ** DB Stats ** Uptime(secs): 6.4 total, 6.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 154.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.287 H:M:S, 20.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 157.74 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.287 H:M:S, 20.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/05-20:34:22 ... thread 0: (1000000,2000000) ops and (245485.5,194023.7) ops/second in (4.073561,10.308021) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/11 3.82 GB 162.7 1.4 0.0 1.4 3.2 1.8 0.0 1.8 85.3 196.1 16.96 12.52 31 0.547 1468K 7855 0.0 0.0 L1 6/6 725.70 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 895.70 MB 0.9 3.4 1.6 1.8 3.3 1.5 0.0 2.1 137.8 135.3 25.05 17.92 13 1.927 1891K 19K 0.0 0.0 L4 124/0 9.71 GB 1.0 2.0 1.3 0.7 2.0 1.3 0.2 1.5 144.5 142.8 14.16 10.15 7 2.023 1351K 0 0.0 0.0 L5 128/0 10.77 GB 0.1 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 292/17 25.88 GB 0.0 6.8 2.9 3.9 8.5 4.6 1.5 4.6 123.6 155.5 56.16 40.59 51 1.101 4711K 27K 0.0 0.0 Int 0/0 0.00 KB 0.0 6.8 2.9 3.9 8.5 4.6 1.5 4.6 123.6 155.5 56.16 40.59 51 1.101 4711K 27K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.8 2.9 3.9 6.7 2.8 0.0 0.0 149.5 147.5 46.44 34.53 25 1.858 4711K 27K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 194.1 9.72 6.06 26 0.374 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.5 total, 10.3 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 8.53 GB write, 835.52 MB/s write, 6.78 GB read, 664.13 MB/s read, 56.2 seconds Interval compaction: 8.53 GB write, 847.18 MB/s write, 6.78 GB read, 673.39 MB/s read, 56.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 16, interval: 16 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 456683 Average: 40.3661 StdDev: 253.56 Min: 1 Median: 1.9252 Max: 36621 Percentiles: P50: 1.93 P75: 2.81 P99: 815.38 P99.9: 2871.87 P99.99: 4141.12 ------------------------------------------------------ [ 0, 1 ] 33908 7.425% 7.425% # ( 1, 2 ] 210155 46.018% 53.443% ######### ( 2, 3 ] 121761 26.662% 80.105% ##### ( 3, 4 ] 25885 5.668% 85.773% # ( 4, 6 ] 24433 5.350% 91.123% # ( 6, 10 ] 5982 1.310% 92.433% ( 10, 15 ] 1509 0.330% 92.763% ( 15, 22 ] 172 0.038% 92.801% ( 22, 34 ] 16 0.004% 92.804% ( 34, 51 ] 6 0.001% 92.806% ( 51, 76 ] 10 0.002% 92.808% ( 76, 110 ] 39 0.009% 92.816% ( 110, 170 ] 2438 0.534% 93.350% ( 170, 250 ] 9447 2.069% 95.419% ( 250, 380 ] 8225 1.801% 97.220% ( 380, 580 ] 5114 1.120% 98.340% ( 580, 870 ] 3716 0.814% 99.153% ( 870, 1300 ] 2023 0.443% 99.596% ( 1300, 1900 ] 880 0.193% 99.789% ( 1900, 2900 ] 522 0.114% 99.903% ( 2900, 4400 ] 479 0.105% 100.008% ( 4400, 6600 ] 117 0.026% 100.034% ( 6600, 9900 ] 19 0.004% 100.038% ( 9900, 14000 ] 4 0.001% 100.039% ( 14000, 22000 ] 5 0.001% 100.040% ( 22000, 33000 ] 3 0.001% 100.041% ( 33000, 50000 ] 3 0.001% 100.041% ** Level 1 read latency histogram (micros): Count: 2165 Average: 670.0430 StdDev: 2219.42 Min: 98 Median: 319.7157 Max: 37892 Percentiles: P50: 319.72 P75: 565.82 P99: 4730.00 P99.9: 37731.67 P99.99: 37892.00 ------------------------------------------------------ ( 76, 110 ] 2 0.092% 0.092% ( 110, 170 ] 185 8.545% 8.637% ## ( 170, 250 ] 622 28.730% 37.367% ###### ( 250, 380 ] 510 23.557% 60.924% ##### ( 380, 580 ] 328 15.150% 76.074% ### ( 580, 870 ] 243 11.224% 87.298% ## ( 870, 1300 ] 139 6.420% 93.718% # ( 1300, 1900 ] 58 2.679% 96.397% # ( 1900, 2900 ] 23 1.062% 97.460% ( 2900, 4400 ] 32 1.478% 98.938% ( 4400, 6600 ] 9 0.416% 99.353% ( 6600, 9900 ] 1 0.046% 99.400% ( 14000, 22000 ] 2 0.092% 99.492% ( 22000, 33000 ] 8 0.370% 99.861% ( 33000, 50000 ] 3 0.139% 100.000% ** Level 2 read latency histogram (micros): Count: 242873 Average: 11.0256 StdDev: 194.31 Min: 1 Median: 2.4901 Max: 32903 Percentiles: P50: 2.49 P75: 3.37 P99: 12.31 P99.9: 856.98 P99.99: 1246.18 ------------------------------------------------------ [ 0, 1 ] 1328 0.547% 0.547% ( 1, 2 ] 71752 29.543% 30.090% ###### ( 2, 3 ] 98658 40.621% 70.711% ######## ( 3, 4 ] 28489 11.730% 82.441% ## ( 4, 6 ] 30734 12.654% 95.095% ### ( 6, 10 ] 8504 3.501% 98.597% # ( 10, 15 ] 2116 0.871% 99.468% ( 15, 22 ] 346 0.142% 99.610% ( 22, 34 ] 33 0.014% 99.624% ( 34, 51 ] 41 0.017% 99.641% ( 51, 76 ] 38 0.016% 99.657% ( 76, 110 ] 12 0.005% 99.662% ( 110, 170 ] 22 0.009% 99.671% ( 170, 250 ] 21 0.009% 99.679% ( 250, 380 ] 59 0.024% 99.704% ( 380, 580 ] 182 0.075% 99.778% ( 580, 870 ] 309 0.127% 99.906% ( 870, 1300 ] 234 0.096% 100.002% ( 1300, 1900 ] 136 0.056% 100.058% ( 1900, 2900 ] 113 0.047% 100.105% ( 2900, 4400 ] 103 0.042% 100.147% ( 4400, 6600 ] 23 0.009% 100.156% ( 6600, 9900 ] 10 0.004% 100.161% ( 9900, 14000 ] 2 0.001% 100.161% ( 14000, 22000 ] 7 0.003% 100.164% ( 22000, 33000 ] 3 0.001% 100.166% ** Level 3 read latency histogram (micros): Count: 657706 Average: 11.0264 StdDev: 127.91 Min: 1 Median: 2.4547 Max: 15164 Percentiles: P50: 2.45 P75: 3.26 P99: 12.82 P99.9: 522.24 P99.99: 667.98 ------------------------------------------------------ [ 0, 1 ] 5362 0.815% 0.815% ( 1, 2 ] 202925 30.853% 31.669% ###### ( 2, 3 ] 265177 40.318% 71.987% ######## ( 3, 4 ] 77103 11.723% 83.710% ## ( 4, 6 ] 77870 11.840% 95.550% ## ( 6, 10 ] 20117 3.059% 98.608% # ( 10, 15 ] 4569 0.695% 99.303% ( 15, 22 ] 519 0.079% 99.382% ( 22, 34 ] 49 0.007% 99.390% ( 34, 51 ] 80 0.012% 99.402% ( 51, 76 ] 136 0.021% 99.422% ( 76, 110 ] 27 0.004% 99.426% ( 110, 170 ] 298 0.045% 99.472% ( 170, 250 ] 1003 0.152% 99.624% ( 250, 380 ] 1090 0.166% 99.790% ( 380, 580 ] 1017 0.155% 99.945% ( 580, 870 ] 983 0.149% 100.094% ( 870, 1300 ] 677 0.103% 100.197% ( 1300, 1900 ] 365 0.055% 100.253% ( 1900, 2900 ] 270 0.041% 100.294% ( 2900, 4400 ] 283 0.043% 100.337% ( 4400, 6600 ] 43 0.007% 100.343% ( 6600, 9900 ] 19 0.003% 100.346% ( 9900, 14000 ] 4 0.001% 100.347% ( 14000, 22000 ] 1 0.000% 100.347% ** Level 4 read latency histogram (micros): Count: 207965 Average: 71.9171 StdDev: 335.89 Min: 0 Median: 2.2036 Max: 26052 Percentiles: P50: 2.20 P75: 3.41 P99: 1216.73 P99.9: 4163.79 P99.99: 5994.01 ------------------------------------------------------ [ 0, 1 ] 7577 3.643% 3.643% # ( 1, 2 ] 84579 40.670% 44.313% ######## ( 2, 3 ] 58095 27.935% 72.248% ###### ( 3, 4 ] 14060 6.761% 79.009% # ( 4, 6 ] 11997 5.769% 84.778% # ( 6, 10 ] 4412 2.122% 86.899% ( 10, 15 ] 1088 0.523% 87.422% ( 15, 22 ] 135 0.065% 87.487% ( 22, 34 ] 12 0.006% 87.493% ( 34, 51 ] 9 0.004% 87.497% ( 51, 76 ] 17 0.008% 87.506% ( 76, 110 ] 25 0.012% 87.518% ( 110, 170 ] 2144 1.031% 88.549% ( 170, 250 ] 7247 3.485% 92.033% # ( 250, 380 ] 6222 2.992% 95.025% # ( 380, 580 ] 3978 1.913% 96.938% ( 580, 870 ] 2950 1.419% 98.356% ( 870, 1300 ] 1661 0.799% 99.155% ( 1300, 1900 ] 708 0.340% 99.496% ( 1900, 2900 ] 457 0.220% 99.715% ( 2900, 4400 ] 457 0.220% 99.935% ( 4400, 6600 ] 159 0.076% 100.012% ( 6600, 9900 ] 32 0.015% 100.027% ( 9900, 14000 ] 6 0.003% 100.030% ( 14000, 22000 ] 2 0.001% 100.031% ( 22000, 33000 ] 3 0.001% 100.032% ** Level 5 read latency histogram (micros): Count: 23216 Average: 554.7013 StdDev: 762.98 Min: 93 Median: 325.3152 Max: 18898 Percentiles: P50: 325.32 P75: 578.82 P99: 4216.85 P99.9: 7827.14 P99.99: 13560.48 ------------------------------------------------------ ( 76, 110 ] 44 0.190% 0.190% ( 110, 170 ] 1928 8.305% 8.494% ## ( 170, 250 ] 6511 28.045% 36.539% ###### ( 250, 380 ] 5394 23.234% 59.773% ##### ( 380, 580 ] 3556 15.317% 75.090% ### ( 580, 870 ] 2619 11.281% 86.371% ## ( 870, 1300 ] 1535 6.612% 92.983% # ( 1300, 1900 ] 651 2.804% 95.787% # ( 1900, 2900 ] 378 1.628% 97.416% ( 2900, 4400 ] 419 1.805% 99.220% ( 4400, 6600 ] 147 0.633% 99.854% ( 6600, 9900 ] 29 0.125% 99.978% ( 9900, 14000 ] 3 0.013% 99.991% ( 14000, 22000 ] 2 0.009% 100.000% ** DB Stats ** Uptime(secs): 10.5 total, 4.1 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 188.26 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.564 H:M:S, 15.0 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 241.60 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.277 H:M:S, 6.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/05-20:34:29 ... thread 0: (1000000,3000000) ops and (147065.3,175359.4) ops/second in (6.799700,17.107721) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/11 4.80 GB 206.4 2.7 0.0 2.7 5.5 2.8 0.0 2.0 105.0 214.2 26.35 20.96 52 0.507 2810K 14K 0.0 0.0 L1 6/6 725.70 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 16/0 895.70 MB 0.9 3.4 1.6 1.8 3.3 1.5 0.0 2.1 137.8 135.3 25.05 17.92 13 1.927 1891K 19K 0.0 0.0 L4 124/0 9.71 GB 1.0 2.0 1.3 0.7 2.0 1.3 0.2 1.5 144.5 142.8 14.16 10.15 7 2.023 1351K 0 0.0 0.0 L5 128/0 10.77 GB 0.1 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 293/17 26.85 GB 0.0 8.1 2.9 5.2 10.8 5.6 1.5 3.8 126.1 168.6 65.56 49.03 72 0.911 6053K 34K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.1 2.9 5.2 10.8 5.6 1.5 3.8 126.1 168.6 65.56 49.03 72 0.911 6053K 34K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.1 2.9 5.2 8.0 2.8 0.0 0.0 158.5 156.5 52.15 40.09 30 1.738 6053K 34K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 215.7 13.41 8.94 42 0.319 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.3 total, 17.1 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.80 GB write, 640.76 MB/s write, 8.07 GB read, 478.97 MB/s read, 65.6 seconds Interval compaction: 10.80 GB write, 646.15 MB/s write, 8.07 GB read, 482.99 MB/s read, 65.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 8, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 8, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 16, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 8, total-stops: 16, interval: 24 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 831535 Average: 37.5621 StdDev: 197.94 Min: 0 Median: 1.8164 Max: 36621 Percentiles: P50: 1.82 P75: 2.72 P99: 576.36 P99.9: 1820.18 P99.99: 3170.87 ------------------------------------------------------ [ 0, 1 ] 76606 9.213% 9.213% ## ( 1, 2 ] 415457 49.963% 59.175% ########## ( 2, 3 ] 182038 21.892% 81.067% #### ( 3, 4 ] 33628 4.044% 85.111% # ( 4, 6 ] 31387 3.775% 88.886% # ( 6, 10 ] 7545 0.907% 89.793% ( 10, 15 ] 1859 0.224% 90.017% ( 15, 22 ] 225 0.027% 90.044% ( 22, 34 ] 41 0.005% 90.049% ( 34, 51 ] 9 0.001% 90.050% ( 51, 76 ] 14 0.002% 90.051% ( 76, 110 ] 182 0.022% 90.073% ( 110, 170 ] 10491 1.262% 91.335% ( 170, 250 ] 37022 4.452% 95.787% # ( 250, 380 ] 18815 2.263% 98.050% ( 380, 580 ] 8048 0.968% 99.018% ( 580, 870 ] 4503 0.542% 99.559% ( 870, 1300 ] 2062 0.248% 99.807% ( 1300, 1900 ] 891 0.107% 99.914% ( 1900, 2900 ] 541 0.065% 99.979% ( 2900, 4400 ] 492 0.059% 100.039% ( 4400, 6600 ] 117 0.014% 100.053% ( 6600, 9900 ] 19 0.002% 100.055% ( 9900, 14000 ] 4 0.000% 100.055% ( 14000, 22000 ] 5 0.001% 100.056% ( 22000, 33000 ] 3 0.000% 100.056% ( 33000, 50000 ] 3 0.000% 100.057% ** Level 1 read latency histogram (micros): Count: 5131 Average: 422.7478 StdDev: 1459.91 Min: 98 Median: 237.4084 Max: 37892 Percentiles: P50: 237.41 P75: 357.80 P99: 3113.18 P99.9: 30069.88 P99.99: 37892.00 ------------------------------------------------------ ( 76, 110 ] 8 0.156% 0.156% ( 110, 170 ] 649 12.649% 12.805% ### ( 170, 250 ] 2265 44.143% 56.948% ######### ( 250, 380 ] 1117 21.770% 78.718% #### ( 380, 580 ] 522 10.173% 88.891% ## ( 580, 870 ] 292 5.691% 94.582% # ( 870, 1300 ] 140 2.729% 97.310% # ( 1300, 1900 ] 58 1.130% 98.441% ( 1900, 2900 ] 24 0.468% 98.909% ( 2900, 4400 ] 33 0.643% 99.552% ( 4400, 6600 ] 9 0.175% 99.727% ( 6600, 9900 ] 1 0.019% 99.747% ( 14000, 22000 ] 2 0.039% 99.786% ( 22000, 33000 ] 8 0.156% 99.942% ( 33000, 50000 ] 3 0.058% 100.000% ** Level 2 read latency histogram (micros): Count: 242873 Average: 11.0256 StdDev: 194.31 Min: 1 Median: 2.4901 Max: 32903 Percentiles: P50: 2.49 P75: 3.37 P99: 12.31 P99.9: 856.98 P99.99: 1246.18 ------------------------------------------------------ [ 0, 1 ] 1328 0.547% 0.547% ( 1, 2 ] 71752 29.543% 30.090% ###### ( 2, 3 ] 98658 40.621% 70.711% ######## ( 3, 4 ] 28489 11.730% 82.441% ## ( 4, 6 ] 30734 12.654% 95.095% ### ( 6, 10 ] 8504 3.501% 98.597% # ( 10, 15 ] 2116 0.871% 99.468% ( 15, 22 ] 346 0.142% 99.610% ( 22, 34 ] 33 0.014% 99.624% ( 34, 51 ] 41 0.017% 99.641% ( 51, 76 ] 38 0.016% 99.657% ( 76, 110 ] 12 0.005% 99.662% ( 110, 170 ] 22 0.009% 99.671% ( 170, 250 ] 21 0.009% 99.679% ( 250, 380 ] 59 0.024% 99.704% ( 380, 580 ] 182 0.075% 99.778% ( 580, 870 ] 309 0.127% 99.906% ( 870, 1300 ] 234 0.096% 100.002% ( 1300, 1900 ] 136 0.056% 100.058% ( 1900, 2900 ] 113 0.047% 100.105% ( 2900, 4400 ] 103 0.042% 100.147% ( 4400, 6600 ] 23 0.009% 100.156% ( 6600, 9900 ] 10 0.004% 100.161% ( 9900, 14000 ] 2 0.001% 100.161% ( 14000, 22000 ] 7 0.003% 100.164% ( 22000, 33000 ] 3 0.001% 100.166% ** Level 3 read latency histogram (micros): Count: 662131 Average: 12.5528 StdDev: 129.08 Min: 1 Median: 2.4630 Max: 15164 Percentiles: P50: 2.46 P75: 3.30 P99: 182.20 P99.9: 544.78 P99.99: 681.74 ------------------------------------------------------ [ 0, 1 ] 5362 0.810% 0.810% ( 1, 2 ] 202925 30.647% 31.457% ###### ( 2, 3 ] 265177 40.049% 71.506% ######## ( 3, 4 ] 77103 11.645% 83.151% ## ( 4, 6 ] 77870 11.761% 94.911% ## ( 6, 10 ] 20117 3.038% 97.949% # ( 10, 15 ] 4569 0.690% 98.640% ( 15, 22 ] 519 0.078% 98.718% ( 22, 34 ] 49 0.007% 98.725% ( 34, 51 ] 80 0.012% 98.737% ( 51, 76 ] 136 0.021% 98.758% ( 76, 110 ] 46 0.007% 98.765% ( 110, 170 ] 1038 0.157% 98.922% ( 170, 250 ] 3400 0.513% 99.435% ( 250, 380 ] 2020 0.305% 99.740% ( 380, 580 ] 1284 0.194% 99.934% ( 580, 870 ] 1054 0.159% 100.093% ( 870, 1300 ] 677 0.102% 100.196% ( 1300, 1900 ] 366 0.055% 100.251% ( 1900, 2900 ] 270 0.041% 100.292% ( 2900, 4400 ] 283 0.043% 100.334% ( 4400, 6600 ] 43 0.006% 100.341% ( 6600, 9900 ] 19 0.003% 100.344% ( 9900, 14000 ] 4 0.001% 100.344% ( 14000, 22000 ] 1 0.000% 100.344% ** Level 4 read latency histogram (micros): Count: 244515 Average: 96.1126 StdDev: 317.90 Min: 0 Median: 2.5145 Max: 26052 Percentiles: P50: 2.51 P75: 115.21 P99: 1128.73 P99.9: 4046.91 P99.99: 5943.44 ------------------------------------------------------ [ 0, 1 ] 7577 3.099% 3.099% # ( 1, 2 ] 84677 34.631% 37.729% ####### ( 2, 3 ] 58316 23.850% 61.579% ##### ( 3, 4 ] 14140 5.783% 67.362% # ( 4, 6 ] 12146 4.967% 72.329% # ( 6, 10 ] 4426 1.810% 74.139% ( 10, 15 ] 1095 0.448% 74.587% ( 15, 22 ] 138 0.056% 74.644% ( 22, 34 ] 13 0.005% 74.649% ( 34, 51 ] 9 0.004% 74.653% ( 51, 76 ] 17 0.007% 74.660% ( 76, 110 ] 108 0.044% 74.704% ( 110, 170 ] 8348 3.414% 78.118% # ( 170, 250 ] 27032 11.055% 89.173% ## ( 250, 380 ] 13535 5.535% 94.709% # ( 380, 580 ] 6021 2.462% 97.171% ( 580, 870 ] 3458 1.414% 98.585% ( 870, 1300 ] 1685 0.689% 99.274% ( 1300, 1900 ] 715 0.292% 99.567% ( 1900, 2900 ] 462 0.189% 99.756% ( 2900, 4400 ] 461 0.189% 99.944% ( 4400, 6600 ] 159 0.065% 100.009% ( 6600, 9900 ] 32 0.013% 100.022% ( 9900, 14000 ] 6 0.002% 100.025% ( 14000, 22000 ] 2 0.001% 100.026% ( 22000, 33000 ] 3 0.001% 100.027% ** Level 5 read latency histogram (micros): Count: 54820 Average: 372.0297 StdDev: 527.20 Min: 93 Median: 236.4950 Max: 18898 Percentiles: P50: 236.50 P75: 357.29 P99: 3094.79 P99.9: 6288.41 P99.99: 9845.15 ------------------------------------------------------ ( 76, 110 ] 145 0.265% 0.265% ( 110, 170 ] 7442 13.575% 13.840% ### ( 170, 250 ] 23849 43.504% 57.344% ######### ( 250, 380 ] 11728 21.394% 78.738% #### ( 380, 580 ] 5377 9.808% 88.546% ## ( 580, 870 ] 3079 5.617% 94.163% # ( 870, 1300 ] 1551 2.829% 96.992% # ( 1300, 1900 ] 659 1.202% 98.194% ( 1900, 2900 ] 387 0.706% 98.900% ( 2900, 4400 ] 422 0.770% 99.670% ( 4400, 6600 ] 147 0.268% 99.938% ( 6600, 9900 ] 29 0.053% 99.991% ( 9900, 14000 ] 3 0.005% 99.996% ( 14000, 22000 ] 2 0.004% 100.000% ** DB Stats ** Uptime(secs): 17.3 total, 6.8 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 171.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.176 H:M:S, 24.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 144.74 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.612 H:M:S, 38.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 399.103 micros/op 9973 ops/sec 20.254 seconds 201996 operations; 9.7 MB/s (49999 of 49999 found) Microseconds per read: Count: 201996 Average: 399.1116 StdDev: 944.96 Min: 1 Median: 274.8007 Max: 58781 Percentiles: P50: 274.80 P75: 403.85 P99: 2397.33 P99.9: 17244.98 P99.99: 32977.36 ------------------------------------------------------ [ 0, 1 ] 1 0.000% 0.000% ( 1, 2 ] 27 0.013% 0.014% ( 2, 3 ] 103 0.051% 0.065% ( 3, 4 ] 101 0.050% 0.115% ( 4, 6 ] 174 0.086% 0.201% ( 6, 10 ] 332 0.164% 0.365% ( 10, 15 ] 3074 1.522% 1.887% ( 15, 22 ] 2313 1.145% 3.032% ( 22, 34 ] 3053 1.511% 4.544% ( 34, 51 ] 1438 0.712% 5.256% ( 51, 76 ] 326 0.161% 5.417% ( 76, 110 ] 70 0.035% 5.452% ( 110, 170 ] 8800 4.357% 9.808% # ( 170, 250 ] 70085 34.696% 44.504% ####### ( 250, 380 ] 58189 28.807% 73.311% ###### ( 380, 580 ] 28609 14.163% 87.475% ### ( 580, 870 ] 14615 7.235% 94.710% # ( 870, 1300 ] 6018 2.979% 97.689% # ( 1300, 1900 ] 2097 1.038% 98.727% ( 1900, 2900 ] 1108 0.549% 99.276% ( 2900, 4400 ] 883 0.437% 99.713% ( 4400, 6600 ] 214 0.106% 99.819% ( 6600, 9900 ] 64 0.032% 99.850% ( 9900, 14000 ] 42 0.021% 99.871% ( 14000, 22000 ] 143 0.071% 99.942% ( 22000, 33000 ] 97 0.048% 99.990% ( 33000, 50000 ] 16 0.008% 99.998% ( 50000, 75000 ] 4 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3447007 rocksdb.bloom.filter.full.positive COUNT : 234448 rocksdb.bloom.filter.full.true.positive COUNT : 201080 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 916 rocksdb.memtable.miss COUNT : 201080 rocksdb.l0.hit COUNT : 38458 rocksdb.l1.hit COUNT : 6086 rocksdb.l2andup.hit COUNT : 156536 rocksdb.compaction.key.drop.new COUNT : 120253 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3073760 rocksdb.number.keys.read COUNT : 201996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3172120320 rocksdb.bytes.read COUNT : 201996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 401 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 6965205 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 9718312057 rocksdb.compact.write.bytes COUNT : 9586460251 rocksdb.flush.write.bytes COUNT : 3165673789 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 22 rocksdb.number.superversion_acquires COUNT : 525 rocksdb.number.superversion_releases COUNT : 498 rocksdb.number.superversion_cleanups COUNT : 127 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3145164800 rocksdb.memtable.garbage.bytes.at.flush COUNT : 5821440 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 9565101908 rocksdb.non.last.level.read.count COUNT : 2278689 rocksdb.block.checksum.compute.count COUNT : 2591544 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 273.359022 P95 : 909.815877 P99 : 2395.963801 P100 : 58779.000000 COUNT : 201996 SUM : 80459087 rocksdb.db.write.micros P50 : 1.740099 P95 : 5.627478 P99 : 9.372095 P100 : 598279.000000 COUNT : 3073760 SUM : 15953499 rocksdb.compaction.times.micros P50 : 1620000.000000 P95 : 3600000.000000 P99 : 4319070.000000 P100 : 4319070.000000 COUNT : 30 SUM : 52146892 rocksdb.compaction.times.cpu_micros P50 : 1364705.882353 P95 : 2400000.000000 P99 : 3271495.000000 P100 : 3271495.000000 COUNT : 30 SUM : 40094107 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1471.428571 P95 : 4325.000000 P99 : 16069.000000 P100 : 16069.000000 COUNT : 44 SUM : 91672 rocksdb.compaction.outfile.sync.micros P50 : 2525.000000 P95 : 9771.666667 P99 : 11105.000000 P100 : 11105.000000 COUNT : 87 SUM : 295648 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 346.451613 P95 : 1156.666667 P99 : 4840.000000 P100 : 5197.000000 COUNT : 80 SUM : 39394 rocksdb.table.open.io.micros P50 : 11190.740741 P95 : 24681.250000 P99 : 66645.833333 P100 : 69482.000000 COUNT : 401 SUM : 4747757 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.740059 P95 : 10.320682 P99 : 294.278816 P100 : 15647.000000 COUNT : 2354375 SUM : 40142862 rocksdb.read.block.get.micros P50 : 225.225809 P95 : 778.329638 P99 : 2063.885542 P100 : 38205.000000 COUNT : 237077 SUM : 72880962 rocksdb.write.raw.block.micros P50 : 0.553384 P95 : 1.629590 P99 : 4.090720 P100 : 28904.000000 COUNT : 3091316 SUM : 10767881 rocksdb.numfiles.in.singlecompaction P50 : 3.184211 P95 : 6.600000 P99 : 13.000000 P100 : 13.000000 COUNT : 37 SUM : 141 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.563232 P95 : 1142.731785 P99 : 2213.717728 P100 : 598072.000000 COUNT : 40573 SUM : 6967935 rocksdb.sst.read.micros P50 : 2.237581 P95 : 246.465183 P99 : 667.016977 P100 : 37892.000000 COUNT : 2279202 SUM : 102581424 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 201996 SUM : 201996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3073760 SUM : 3172120320 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 268571.428571 P95 : 744000.000000 P99 : 866710.000000 P100 : 866710.000000 COUNT : 44 SUM : 13842514 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 106 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0