[2024-07-15 15:54:37.221888] Starting SPDK v24.09-pre git sha1 a95bbf233 / DPDK 24.03.0 initialization... [2024-07-15 15:54:37.222207] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid2256872 ] Set seed to 1721051677214633 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 15:54:37.367533] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 15:54:37.590799] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 15:54:37.590850] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 2256872' to capture a snapshot of events at runtime. [2024-07-15 15:54:37.590863] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 15:54:37.590876] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 15:54:37.590886] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid2256872 for offline analysis/debug. [2024-07-15 15:54:37.590988] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 15:54:41 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/15-15:54:46 ... thread 0: (1000000,1000000) ops and (194044.2,194044.2) ops/second in (5.153465,5.153465) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/9 1.31 GB 75.2 0.3 0.0 0.3 1.2 0.9 0.0 1.4 46.7 177.0 6.74 3.75 13 0.518 319K 1835 0.0 0.0 L1 8/8 1.01 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 2/2 213.90 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/2 1.05 GB 0.9 2.1 1.2 0.8 2.0 1.2 0.0 1.6 145.9 141.2 14.44 9.97 10 1.444 1128K 6067 0.0 0.0 L4 127/0 9.60 GB 1.0 2.2 1.0 1.2 2.0 0.8 0.9 2.1 151.6 138.1 14.85 9.69 9 1.650 518K 433 0.0 0.0 L5 140/0 10.83 GB 0.1 0.0 0.0 0.0 0.0 0.0 1.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 309/21 24.00 GB 0.0 4.6 2.2 2.4 5.2 2.8 2.5 6.0 129.7 146.6 36.03 23.41 32 1.126 1966K 8335 0.0 0.0 Int 0/0 0.00 KB 0.0 4.6 2.2 2.4 5.2 2.8 2.5 6.0 129.7 146.6 36.03 23.41 32 1.126 1966K 8335 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 4.6 2.2 2.4 4.3 1.9 0.0 0.0 149.4 140.7 31.27 20.97 20 1.564 1966K 8335 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 185.1 4.76 2.43 12 0.396 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 5.2 total, 5.2 interval Flush(GB): cumulative 0.860, interval 0.860 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.16 GB write, 1007.08 MB/s write, 4.56 GB read, 890.94 MB/s read, 36.0 seconds Interval compaction: 5.16 GB write, 1024.41 MB/s write, 4.56 GB read, 906.26 MB/s read, 36.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 5, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 5, interval: 5 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 90914 Average: 83.7663 StdDev: 402.08 Min: 1 Median: 2.4063 Max: 34230 Percentiles: P50: 2.41 P75: 3.95 P99: 1473.53 P99.9: 4306.36 P99.99: 6399.98 ------------------------------------------------------ [ 0, 1 ] 1509 1.660% 1.660% ( 1, 2 ] 33952 37.345% 39.005% ####### ( 2, 3 ] 24605 27.064% 66.069% ##### ( 3, 4 ] 8588 9.446% 75.515% ## ( 4, 6 ] 9650 10.614% 86.130% ## ( 6, 10 ] 2608 2.869% 88.998% # ( 10, 15 ] 586 0.645% 89.643% ( 15, 22 ] 70 0.077% 89.720% ( 22, 34 ] 7 0.008% 89.728% ( 34, 51 ] 14 0.015% 89.743% ( 51, 76 ] 6 0.007% 89.750% ( 76, 110 ] 6 0.007% 89.756% ( 110, 170 ] 167 0.184% 89.940% ( 170, 250 ] 907 0.998% 90.938% ( 250, 380 ] 1492 1.641% 92.579% ( 380, 580 ] 2567 2.824% 95.402% # ( 580, 870 ] 2287 2.516% 97.918% # ( 870, 1300 ] 913 1.004% 98.922% ( 1300, 1900 ] 245 0.269% 99.192% ( 1900, 2900 ] 330 0.363% 99.555% ( 2900, 4400 ] 335 0.368% 99.923% ( 4400, 6600 ] 67 0.074% 99.997% ( 6600, 9900 ] 5 0.005% 100.002% ( 9900, 14000 ] 5 0.005% 100.008% ( 14000, 22000 ] 3 0.003% 100.011% ( 22000, 33000 ] 1 0.001% 100.012% ( 33000, 50000 ] 1 0.001% 100.013% ** Level 1 read latency histogram (micros): Count: 177464 Average: 14.0426 StdDev: 192.99 Min: 1 Median: 2.2776 Max: 27060 Percentiles: P50: 2.28 P75: 3.30 P99: 53.57 P99.9: 2641.54 P99.99: 5903.86 ------------------------------------------------------ [ 0, 1 ] 1594 0.898% 0.898% ( 1, 2 ] 72332 40.759% 41.657% ######## ( 2, 3 ] 53337 30.055% 71.712% ###### ( 3, 4 ] 19681 11.090% 82.802% ## ( 4, 6 ] 21145 11.915% 94.717% ## ( 6, 10 ] 5987 3.374% 98.091% # ( 10, 15 ] 1359 0.766% 98.857% ( 15, 22 ] 181 0.102% 98.959% ( 22, 34 ] 27 0.015% 98.974% ( 34, 51 ] 44 0.025% 98.999% ( 51, 76 ] 23 0.013% 99.012% ( 76, 110 ] 10 0.006% 99.017% ( 110, 170 ] 31 0.017% 99.035% ( 170, 250 ] 119 0.067% 99.102% ( 250, 380 ] 261 0.147% 99.249% ( 380, 580 ] 412 0.232% 99.481% ( 580, 870 ] 366 0.206% 99.687% ( 870, 1300 ] 187 0.105% 99.793% ( 1300, 1900 ] 106 0.060% 99.852% ( 1900, 2900 ] 114 0.064% 99.917% ( 2900, 4400 ] 120 0.068% 99.984% ( 4400, 6600 ] 15 0.008% 99.993% ( 6600, 9900 ] 5 0.003% 99.995% ( 9900, 14000 ] 6 0.003% 99.999% ( 14000, 22000 ] 5 0.003% 100.002% ( 22000, 33000 ] 1 0.001% 100.002% ** Level 2 read latency histogram (micros): Count: 229153 Average: 9.3219 StdDev: 158.79 Min: 1 Median: 2.2238 Max: 28959 Percentiles: P50: 2.22 P75: 3.20 P99: 13.19 P99.9: 841.99 P99.99: 2255.50 ------------------------------------------------------ [ 0, 1 ] 2528 1.103% 1.103% ( 1, 2 ] 96944 42.305% 43.409% ######## ( 2, 3 ] 67486 29.450% 72.859% ###### ( 3, 4 ] 24983 10.902% 83.761% ## ( 4, 6 ] 26230 11.447% 95.208% ## ( 6, 10 ] 7596 3.315% 98.522% # ( 10, 15 ] 1715 0.748% 99.271% ( 15, 22 ] 204 0.089% 99.360% ( 22, 34 ] 23 0.010% 99.370% ( 34, 51 ] 66 0.029% 99.399% ( 51, 76 ] 47 0.021% 99.419% ( 76, 110 ] 5 0.002% 99.421% ( 110, 170 ] 28 0.012% 99.434% ( 170, 250 ] 153 0.067% 99.500% ( 250, 380 ] 380 0.166% 99.666% ( 380, 580 ] 338 0.147% 99.814% ( 580, 870 ] 219 0.096% 99.909% ( 870, 1300 ] 94 0.041% 99.950% ( 1300, 1900 ] 63 0.027% 99.978% ( 1900, 2900 ] 79 0.034% 100.012% ( 2900, 4400 ] 64 0.028% 100.040% ( 4400, 6600 ] 25 0.011% 100.051% ( 6600, 9900 ] 4 0.002% 100.053% ( 9900, 14000 ] 3 0.001% 100.054% ( 14000, 22000 ] 4 0.002% 100.056% ( 22000, 33000 ] 2 0.001% 100.057% ** Level 3 read latency histogram (micros): Count: 384605 Average: 11.7146 StdDev: 157.08 Min: 1 Median: 2.3217 Max: 24776 Percentiles: P50: 2.32 P75: 3.32 P99: 14.09 P99.9: 986.16 P99.99: 1807.70 ------------------------------------------------------ [ 0, 1 ] 2871 0.746% 0.746% ( 1, 2 ] 150323 39.085% 39.832% ######## ( 2, 3 ] 121581 31.612% 71.443% ###### ( 3, 4 ] 43297 11.258% 82.701% ## ( 4, 6 ] 47701 12.403% 95.104% ## ( 6, 10 ] 12565 3.267% 98.371% # ( 10, 15 ] 2958 0.769% 99.140% ( 15, 22 ] 356 0.093% 99.232% ( 22, 34 ] 56 0.015% 99.247% ( 34, 51 ] 102 0.027% 99.273% ( 51, 76 ] 59 0.015% 99.289% ( 76, 110 ] 15 0.004% 99.293% ( 110, 170 ] 58 0.015% 99.308% ( 170, 250 ] 216 0.056% 99.364% ( 250, 380 ] 478 0.124% 99.488% ( 380, 580 ] 823 0.214% 99.702% ( 580, 870 ] 679 0.177% 99.879% ( 870, 1300 ] 305 0.079% 99.958% ( 1300, 1900 ] 146 0.038% 99.996% ( 1900, 2900 ] 192 0.050% 100.046% ( 2900, 4400 ] 213 0.055% 100.101% ( 4400, 6600 ] 46 0.012% 100.113% ( 6600, 9900 ] 8 0.002% 100.115% ( 9900, 14000 ] 3 0.001% 100.116% ( 14000, 22000 ] 3 0.001% 100.117% ( 22000, 33000 ] 2 0.001% 100.117% ** Level 4 read latency histogram (micros): Count: 302819 Average: 33.3407 StdDev: 254.20 Min: 0 Median: 2.3493 Max: 27571 Percentiles: P50: 2.35 P75: 3.48 P99: 758.58 P99.9: 2982.94 P99.99: 3685.36 ------------------------------------------------------ [ 0, 1 ] 2157 0.712% 0.712% ( 1, 2 ] 117214 38.708% 39.420% ######## ( 2, 3 ] 91712 30.286% 69.706% ###### ( 3, 4 ] 33261 10.984% 80.690% ## ( 4, 6 ] 36214 11.959% 92.649% ## ( 6, 10 ] 10013 3.307% 95.955% # ( 10, 15 ] 2238 0.739% 96.694% ( 15, 22 ] 302 0.100% 96.794% ( 22, 34 ] 63 0.021% 96.815% ( 34, 51 ] 112 0.037% 96.852% ( 51, 76 ] 64 0.021% 96.873% ( 76, 110 ] 27 0.009% 96.882% ( 110, 170 ] 169 0.056% 96.938% ( 170, 250 ] 769 0.254% 97.192% ( 250, 380 ] 1524 0.503% 97.695% ( 380, 580 ] 2557 0.844% 98.539% ( 580, 870 ] 2265 0.748% 99.287% ( 870, 1300 ] 923 0.305% 99.592% ( 1300, 1900 ] 388 0.128% 99.720% ( 1900, 2900 ] 512 0.169% 99.889% ( 2900, 4400 ] 582 0.192% 100.082% ( 4400, 6600 ] 119 0.039% 100.121% ( 6600, 9900 ] 22 0.007% 100.128% ( 9900, 14000 ] 2 0.001% 100.129% ( 14000, 22000 ] 1 0.000% 100.129% ( 22000, 33000 ] 1 0.000% 100.129% ** Level 5 read latency histogram (micros): Count: 8023 Average: 889.7496 StdDev: 1147.41 Min: 95 Median: 565.5037 Max: 32891 Percentiles: P50: 565.50 P75: 863.33 P99: 5019.25 P99.9: 9068.68 P99.99: 28043.67 ------------------------------------------------------ ( 76, 110 ] 5 0.062% 0.062% ( 110, 170 ] 115 1.433% 1.496% ( 170, 250 ] 661 8.239% 9.735% ## ( 250, 380 ] 1343 16.739% 26.474% ### ( 380, 580 ] 2035 25.365% 51.838% ##### ( 580, 870 ] 1902 23.707% 75.545% ##### ( 870, 1300 ] 792 9.872% 85.417% ## ( 1300, 1900 ] 304 3.789% 89.206% # ( 1900, 2900 ] 366 4.562% 93.768% # ( 2900, 4400 ] 395 4.923% 98.691% # ( 4400, 6600 ] 88 1.097% 99.788% ( 6600, 9900 ] 12 0.150% 99.938% ( 9900, 14000 ] 1 0.012% 99.950% ( 14000, 22000 ] 1 0.012% 99.963% ( 22000, 33000 ] 4 0.050% 100.012% ** DB Stats ** Uptime(secs): 5.2 total, 5.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 187.65 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.700 H:M:S, 13.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 190.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.700 H:M:S, 13.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 4 num-running-flushes: 1 2024/07/15-15:54:51 ... thread 0: (1000000,2000000) ops and (193638.9,193841.3) ops/second in (5.164252,10.317717) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/8 2.29 GB 171.6 1.3 0.0 1.3 3.2 1.8 0.0 1.7 56.6 135.6 23.80 11.21 32 0.744 1367K 6885 0.0 0.0 L2 0/0 0.00 KB 0.0 1.1 1.0 0.1 1.1 1.0 0.0 1.1 163.0 160.9 6.84 5.14 1 6.844 1132K 14K 0.0 0.0 L3 21/4 1.14 GB 1.0 3.1 2.0 1.2 3.1 1.9 0.5 1.5 153.7 149.9 20.86 15.25 15 1.390 2079K 8929 0.0 0.0 L4 138/13 9.92 GB 0.9 3.9 1.8 2.1 3.6 1.5 1.1 2.0 161.1 148.8 24.93 17.89 17 1.467 1538K 4453 0.0 0.0 L5 148/0 11.50 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 323/25 24.85 GB 0.0 9.5 4.8 4.6 10.9 6.3 3.9 5.9 126.7 146.1 76.43 49.49 65 1.176 6118K 34K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.5 4.8 4.6 10.9 6.3 3.9 5.9 126.7 146.1 76.43 49.49 65 1.176 6118K 34K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.5 4.8 4.6 9.1 4.4 0.0 0.0 143.4 137.4 67.53 44.18 38 1.777 6118K 34K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 212.0 8.90 5.31 27 0.330 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.4 total, 10.3 interval Flush(GB): cumulative 1.843, interval 1.843 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.90 GB write, 1072.49 MB/s write, 9.46 GB read, 930.53 MB/s read, 76.4 seconds Interval compaction: 10.90 GB write, 1081.71 MB/s write, 9.46 GB read, 938.53 MB/s read, 76.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 6, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 6, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 6, total-stops: 7, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 468435 Average: 44.7893 StdDev: 278.99 Min: 1 Median: 2.0942 Max: 39501 Percentiles: P50: 2.09 P75: 3.29 P99: 783.57 P99.9: 2750.84 P99.99: 3792.31 ------------------------------------------------------ [ 0, 1 ] 13492 2.880% 2.880% # ( 1, 2 ] 209764 44.780% 47.660% ######### ( 2, 3 ] 116349 24.838% 72.498% ##### ( 3, 4 ] 40845 8.719% 81.217% ## ( 4, 6 ] 41006 8.754% 89.971% ## ( 6, 10 ] 9275 1.980% 91.951% ( 10, 15 ] 1756 0.375% 92.326% ( 15, 22 ] 172 0.037% 92.363% ( 22, 34 ] 26 0.006% 92.368% ( 34, 51 ] 73 0.016% 92.384% ( 51, 76 ] 26 0.006% 92.389% ( 76, 110 ] 45 0.010% 92.399% ( 110, 170 ] 1106 0.236% 92.635% ( 170, 250 ] 7537 1.609% 94.244% ( 250, 380 ] 9701 2.071% 96.315% ( 380, 580 ] 8768 1.872% 98.187% ( 580, 870 ] 5430 1.159% 99.346% ( 870, 1300 ] 1666 0.356% 99.702% ( 1300, 1900 ] 410 0.088% 99.789% ( 1900, 2900 ] 613 0.131% 99.920% ( 2900, 4400 ] 555 0.118% 100.038% ( 4400, 6600 ] 98 0.021% 100.059% ( 6600, 9900 ] 17 0.004% 100.063% ( 9900, 14000 ] 10 0.002% 100.065% ( 14000, 22000 ] 13 0.003% 100.068% ( 22000, 33000 ] 9 0.002% 100.070% ( 33000, 50000 ] 2 0.000% 100.070% ** Level 1 read latency histogram (micros): Count: 263499 Average: 11.7439 StdDev: 167.63 Min: 1 Median: 2.1486 Max: 27060 Percentiles: P50: 2.15 P75: 2.99 P99: 156.74 P99.9: 1960.98 P99.99: 4383.54 ------------------------------------------------------ [ 0, 1 ] 2676 1.016% 1.016% ( 1, 2 ] 117475 44.583% 45.598% ######### ( 2, 3 ] 78062 29.625% 75.223% ###### ( 3, 4 ] 25926 9.839% 85.063% ## ( 4, 6 ] 26515 10.063% 95.125% ## ( 6, 10 ] 8110 3.078% 98.203% # ( 10, 15 ] 1688 0.641% 98.844% ( 15, 22 ] 223 0.085% 98.928% ( 22, 34 ] 30 0.011% 98.940% ( 34, 51 ] 48 0.018% 98.958% ( 51, 76 ] 24 0.009% 98.967% ( 76, 110 ] 13 0.005% 98.972% ( 110, 170 ] 95 0.036% 99.008% ( 170, 250 ] 498 0.189% 99.197% ( 250, 380 ] 573 0.217% 99.414% ( 380, 580 ] 553 0.210% 99.624% ( 580, 870 ] 422 0.160% 99.784% ( 870, 1300 ] 191 0.072% 99.857% ( 1300, 1900 ] 106 0.040% 99.897% ( 1900, 2900 ] 123 0.047% 99.944% ( 2900, 4400 ] 123 0.047% 99.991% ( 4400, 6600 ] 15 0.006% 99.996% ( 6600, 9900 ] 5 0.002% 99.998% ( 9900, 14000 ] 6 0.002% 100.000% ( 14000, 22000 ] 7 0.003% 100.003% ( 22000, 33000 ] 1 0.000% 100.003% ** Level 2 read latency histogram (micros): Count: 508434 Average: 8.5589 StdDev: 185.78 Min: 1 Median: 2.2511 Max: 43393 Percentiles: P50: 2.25 P75: 3.14 P99: 11.62 P99.9: 583.80 P99.99: 963.22 ------------------------------------------------------ [ 0, 1 ] 3556 0.699% 0.699% ( 1, 2 ] 210568 41.415% 42.114% ######## ( 2, 3 ] 159697 31.410% 73.524% ###### ( 3, 4 ] 55545 10.925% 84.449% ## ( 4, 6 ] 57682 11.345% 95.794% ## ( 6, 10 ] 15277 3.005% 98.798% # ( 10, 15 ] 3164 0.622% 99.421% ( 15, 22 ] 359 0.071% 99.491% ( 22, 34 ] 83 0.016% 99.508% ( 34, 51 ] 162 0.032% 99.540% ( 51, 76 ] 105 0.021% 99.560% ( 76, 110 ] 14 0.003% 99.563% ( 110, 170 ] 47 0.009% 99.572% ( 170, 250 ] 271 0.053% 99.626% ( 250, 380 ] 690 0.136% 99.761% ( 380, 580 ] 700 0.138% 99.899% ( 580, 870 ] 425 0.084% 99.982% ( 870, 1300 ] 176 0.035% 100.017% ( 1300, 1900 ] 94 0.018% 100.036% ( 1900, 2900 ] 148 0.029% 100.065% ( 2900, 4400 ] 102 0.020% 100.085% ( 4400, 6600 ] 32 0.006% 100.091% ( 6600, 9900 ] 7 0.001% 100.092% ( 9900, 14000 ] 12 0.002% 100.095% ( 14000, 22000 ] 11 0.002% 100.097% ( 22000, 33000 ] 5 0.001% 100.098% ( 33000, 50000 ] 3 0.001% 100.099% ** Level 3 read latency histogram (micros): Count: 614598 Average: 11.1754 StdDev: 148.10 Min: 1 Median: 2.2528 Max: 33387 Percentiles: P50: 2.25 P75: 3.15 P99: 112.44 P99.9: 863.87 P99.99: 1728.97 ------------------------------------------------------ [ 0, 1 ] 5362 0.872% 0.872% ( 1, 2 ] 253293 41.213% 42.085% ######## ( 2, 3 ] 192417 31.308% 73.393% ###### ( 3, 4 ] 65313 10.627% 84.020% ## ( 4, 6 ] 68390 11.128% 95.148% ## ( 6, 10 ] 18740 3.049% 98.197% # ( 10, 15 ] 4074 0.663% 98.860% ( 15, 22 ] 502 0.082% 98.941% ( 22, 34 ] 86 0.014% 98.955% ( 34, 51 ] 174 0.028% 98.984% ( 51, 76 ] 73 0.012% 98.995% ( 76, 110 ] 20 0.003% 98.999% ( 110, 170 ] 197 0.032% 99.031% ( 170, 250 ] 1046 0.170% 99.201% ( 250, 380 ] 1619 0.263% 99.464% ( 380, 580 ] 1631 0.265% 99.730% ( 580, 870 ] 1069 0.174% 99.904% ( 870, 1300 ] 409 0.067% 99.970% ( 1300, 1900 ] 170 0.028% 99.998% ( 1900, 2900 ] 228 0.037% 100.035% ( 2900, 4400 ] 238 0.039% 100.074% ( 4400, 6600 ] 54 0.009% 100.082% ( 6600, 9900 ] 10 0.002% 100.084% ( 9900, 14000 ] 5 0.001% 100.085% ( 14000, 22000 ] 4 0.001% 100.086% ( 22000, 33000 ] 3 0.000% 100.086% ( 33000, 50000 ] 1 0.000% 100.086% ** Level 4 read latency histogram (micros): Count: 684432 Average: 28.5548 StdDev: 245.16 Min: 0 Median: 2.3082 Max: 35297 Percentiles: P50: 2.31 P75: 3.43 P99: 581.96 P99.9: 1942.06 P99.99: 2818.29 ------------------------------------------------------ [ 0, 1 ] 5645 0.825% 0.825% ( 1, 2 ] 274304 40.078% 40.902% ######## ( 2, 3 ] 202039 29.519% 70.422% ###### ( 3, 4 ] 73399 10.724% 81.146% ## ( 4, 6 ] 77168 11.275% 92.420% ## ( 6, 10 ] 21042 3.074% 95.495% # ( 10, 15 ] 4001 0.585% 96.079% ( 15, 22 ] 534 0.078% 96.157% ( 22, 34 ] 97 0.014% 96.172% ( 34, 51 ] 171 0.025% 96.197% ( 51, 76 ] 115 0.017% 96.213% ( 76, 110 ] 57 0.008% 96.222% ( 110, 170 ] 881 0.129% 96.350% ( 170, 250 ] 5000 0.731% 97.081% ( 250, 380 ] 6732 0.984% 98.065% ( 380, 580 ] 6376 0.932% 98.996% ( 580, 870 ] 4239 0.619% 99.615% ( 870, 1300 ] 1416 0.207% 99.822% ( 1300, 1900 ] 504 0.074% 99.896% ( 1900, 2900 ] 703 0.103% 99.999% ( 2900, 4400 ] 733 0.107% 100.106% ( 4400, 6600 ] 146 0.021% 100.127% ( 6600, 9900 ] 28 0.004% 100.131% ( 9900, 14000 ] 15 0.002% 100.133% ( 14000, 22000 ] 16 0.002% 100.136% ( 22000, 33000 ] 10 0.001% 100.137% ( 33000, 50000 ] 2 0.000% 100.137% ** Level 5 read latency histogram (micros): Count: 21789 Average: 615.8616 StdDev: 973.24 Min: 95 Median: 395.7874 Max: 42143 Percentiles: P50: 395.79 P75: 645.14 P99: 4151.31 P99.9: 9205.35 P99.99: 30838.68 ------------------------------------------------------ ( 76, 110 ] 14 0.064% 0.064% ( 110, 170 ] 694 3.185% 3.249% # ( 170, 250 ] 4193 19.244% 22.493% #### ( 250, 380 ] 5594 25.674% 48.167% ##### ( 380, 580 ] 5061 23.227% 71.394% ##### ( 580, 870 ] 3498 16.054% 87.448% ### ( 870, 1300 ] 1210 5.553% 93.001% # ( 1300, 1900 ] 388 1.781% 94.782% ( 1900, 2900 ] 497 2.281% 97.063% ( 2900, 4400 ] 506 2.322% 99.385% ( 4400, 6600 ] 98 0.450% 99.835% ( 6600, 9900 ] 18 0.083% 99.917% ( 9900, 14000 ] 5 0.023% 99.940% ( 14000, 22000 ] 6 0.028% 99.968% ( 22000, 33000 ] 6 0.028% 99.995% ( 33000, 50000 ] 2 0.009% 100.005% ** DB Stats ** Uptime(secs): 10.4 total, 5.2 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 189.10 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.686 H:M:S, 16.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 190.58 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.986 H:M:S, 19.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 7 num-running-flushes: 1 2024/07/15-15:54:55 ... thread 0: (1000000,3000000) ops and (267180.2,213363.5) ops/second in (3.742792,14.060509) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/4 3.27 GB 309.2 2.7 0.0 2.7 5.5 2.8 0.0 1.9 71.9 147.8 37.95 19.61 53 0.716 2772K 14K 0.0 0.0 L2 0/0 0.00 KB 0.0 1.1 1.0 0.1 1.1 1.0 0.0 1.1 163.0 160.9 6.84 5.14 1 6.844 1132K 14K 0.0 0.0 L3 17/0 963.44 MB 1.0 3.1 2.0 1.2 3.1 1.9 0.5 1.5 153.7 149.9 20.86 15.25 15 1.390 2079K 8929 0.0 0.0 L4 138/0 9.66 GB 1.0 5.2 2.0 3.1 4.7 1.5 1.1 2.3 174.1 157.3 30.39 22.83 21 1.447 1746K 4453 0.0 0.0 L5 152/0 11.76 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 324/4 25.63 GB 0.0 12.1 5.0 7.0 14.3 7.3 4.1 5.1 128.5 152.2 96.05 62.83 90 1.067 7731K 42K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.1 5.0 7.0 14.3 7.3 4.1 5.1 128.5 152.2 96.05 62.83 90 1.067 7731K 42K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.1 5.0 7.0 11.4 4.4 0.0 0.0 146.8 139.4 84.08 54.83 47 1.789 7731K 42K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 241.8 11.97 8.00 43 0.278 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 14.2 total, 14.1 interval Flush(GB): cumulative 2.826, interval 2.826 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.27 GB write, 1032.90 MB/s write, 12.06 GB read, 872.33 MB/s read, 96.0 seconds Interval compaction: 14.27 GB write, 1039.42 MB/s write, 12.06 GB read, 877.83 MB/s read, 96.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 6, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 6, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 7, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 6, total-stops: 7, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 753277 Average: 31.9914 StdDev: 223.02 Min: 1 Median: 1.8252 Max: 39501 Percentiles: P50: 1.83 P75: 2.82 P99: 633.17 P99.9: 2123.03 P99.99: 3422.38 ------------------------------------------------------ [ 0, 1 ] 66212 8.790% 8.790% ## ( 1, 2 ] 376184 49.940% 58.730% ########## ( 2, 3 ] 150379 19.963% 78.693% #### ( 3, 4 ] 50885 6.755% 85.448% # ( 4, 6 ] 48269 6.408% 91.856% # ( 6, 10 ] 11569 1.536% 93.392% ( 10, 15 ] 2059 0.273% 93.665% ( 15, 22 ] 201 0.027% 93.692% ( 22, 34 ] 32 0.004% 93.696% ( 34, 51 ] 76 0.010% 93.706% ( 51, 76 ] 28 0.004% 93.710% ( 76, 110 ] 304 0.040% 93.750% ( 110, 170 ] 4959 0.658% 94.408% ( 170, 250 ] 13294 1.765% 96.173% ( 250, 380 ] 11019 1.463% 97.636% ( 380, 580 ] 9256 1.229% 98.865% ( 580, 870 ] 5559 0.738% 99.603% ( 870, 1300 ] 1686 0.224% 99.827% ( 1300, 1900 ] 415 0.055% 99.882% ( 1900, 2900 ] 622 0.083% 99.964% ( 2900, 4400 ] 559 0.074% 100.038% ( 4400, 6600 ] 99 0.013% 100.052% ( 6600, 9900 ] 18 0.002% 100.054% ( 9900, 14000 ] 10 0.001% 100.055% ( 14000, 22000 ] 13 0.002% 100.057% ( 22000, 33000 ] 9 0.001% 100.058% ( 33000, 50000 ] 2 0.000% 100.059% ** Level 1 read latency histogram (micros): Count: 263499 Average: 11.7439 StdDev: 167.63 Min: 1 Median: 2.1486 Max: 27060 Percentiles: P50: 2.15 P75: 2.99 P99: 156.74 P99.9: 1960.98 P99.99: 4383.54 ------------------------------------------------------ [ 0, 1 ] 2676 1.016% 1.016% ( 1, 2 ] 117475 44.583% 45.598% ######### ( 2, 3 ] 78062 29.625% 75.223% ###### ( 3, 4 ] 25926 9.839% 85.063% ## ( 4, 6 ] 26515 10.063% 95.125% ## ( 6, 10 ] 8110 3.078% 98.203% # ( 10, 15 ] 1688 0.641% 98.844% ( 15, 22 ] 223 0.085% 98.928% ( 22, 34 ] 30 0.011% 98.940% ( 34, 51 ] 48 0.018% 98.958% ( 51, 76 ] 24 0.009% 98.967% ( 76, 110 ] 13 0.005% 98.972% ( 110, 170 ] 95 0.036% 99.008% ( 170, 250 ] 498 0.189% 99.197% ( 250, 380 ] 573 0.217% 99.414% ( 380, 580 ] 553 0.210% 99.624% ( 580, 870 ] 422 0.160% 99.784% ( 870, 1300 ] 191 0.072% 99.857% ( 1300, 1900 ] 106 0.040% 99.897% ( 1900, 2900 ] 123 0.047% 99.944% ( 2900, 4400 ] 123 0.047% 99.991% ( 4400, 6600 ] 15 0.006% 99.996% ( 6600, 9900 ] 5 0.002% 99.998% ( 9900, 14000 ] 6 0.002% 100.000% ( 14000, 22000 ] 7 0.003% 100.003% ( 22000, 33000 ] 1 0.000% 100.003% ** Level 2 read latency histogram (micros): Count: 508434 Average: 8.5589 StdDev: 185.78 Min: 1 Median: 2.2511 Max: 43393 Percentiles: P50: 2.25 P75: 3.14 P99: 11.62 P99.9: 583.80 P99.99: 963.22 ------------------------------------------------------ [ 0, 1 ] 3556 0.699% 0.699% ( 1, 2 ] 210568 41.415% 42.114% ######## ( 2, 3 ] 159697 31.410% 73.524% ###### ( 3, 4 ] 55545 10.925% 84.449% ## ( 4, 6 ] 57682 11.345% 95.794% ## ( 6, 10 ] 15277 3.005% 98.798% # ( 10, 15 ] 3164 0.622% 99.421% ( 15, 22 ] 359 0.071% 99.491% ( 22, 34 ] 83 0.016% 99.508% ( 34, 51 ] 162 0.032% 99.540% ( 51, 76 ] 105 0.021% 99.560% ( 76, 110 ] 14 0.003% 99.563% ( 110, 170 ] 47 0.009% 99.572% ( 170, 250 ] 271 0.053% 99.626% ( 250, 380 ] 690 0.136% 99.761% ( 380, 580 ] 700 0.138% 99.899% ( 580, 870 ] 425 0.084% 99.982% ( 870, 1300 ] 176 0.035% 100.017% ( 1300, 1900 ] 94 0.018% 100.036% ( 1900, 2900 ] 148 0.029% 100.065% ( 2900, 4400 ] 102 0.020% 100.085% ( 4400, 6600 ] 32 0.006% 100.091% ( 6600, 9900 ] 7 0.001% 100.092% ( 9900, 14000 ] 12 0.002% 100.095% ( 14000, 22000 ] 11 0.002% 100.097% ( 22000, 33000 ] 5 0.001% 100.098% ( 33000, 50000 ] 3 0.001% 100.099% ** Level 3 read latency histogram (micros): Count: 639920 Average: 12.2382 StdDev: 146.33 Min: 1 Median: 2.2467 Max: 33387 Percentiles: P50: 2.25 P75: 3.16 P99: 230.90 P99.9: 858.48 P99.99: 1723.56 ------------------------------------------------------ [ 0, 1 ] 6248 0.976% 0.976% ( 1, 2 ] 264937 41.402% 42.378% ######## ( 2, 3 ] 197716 30.897% 73.275% ###### ( 3, 4 ] 66928 10.459% 83.734% ## ( 4, 6 ] 69654 10.885% 94.619% ## ( 6, 10 ] 18995 2.968% 97.587% # ( 10, 15 ] 4118 0.644% 98.230% ( 15, 22 ] 506 0.079% 98.309% ( 22, 34 ] 86 0.013% 98.323% ( 34, 51 ] 174 0.027% 98.350% ( 51, 76 ] 73 0.011% 98.362% ( 76, 110 ] 105 0.016% 98.378% ( 110, 170 ] 1627 0.254% 98.632% ( 170, 250 ] 3092 0.483% 99.115% ( 250, 380 ] 2131 0.333% 99.448% ( 380, 580 ] 1828 0.286% 99.734% ( 580, 870 ] 1106 0.173% 99.907% ( 870, 1300 ] 412 0.064% 99.971% ( 1300, 1900 ] 170 0.027% 99.998% ( 1900, 2900 ] 230 0.036% 100.034% ( 2900, 4400 ] 239 0.037% 100.071% ( 4400, 6600 ] 54 0.008% 100.080% ( 6600, 9900 ] 10 0.002% 100.081% ( 9900, 14000 ] 5 0.001% 100.082% ( 14000, 22000 ] 4 0.001% 100.083% ( 22000, 33000 ] 3 0.000% 100.083% ( 33000, 50000 ] 1 0.000% 100.083% ** Level 4 read latency histogram (micros): Count: 820081 Average: 30.4630 StdDev: 228.19 Min: 0 Median: 2.2989 Max: 35297 Percentiles: P50: 2.30 P75: 3.50 P99: 548.56 P99.9: 1643.26 P99.99: 2623.84 ------------------------------------------------------ [ 0, 1 ] 7854 0.958% 0.958% ( 1, 2 ] 332527 40.548% 41.506% ######## ( 2, 3 ] 233037 28.416% 69.922% ###### ( 3, 4 ] 82640 10.077% 79.999% ## ( 4, 6 ] 84873 10.349% 90.349% ## ( 6, 10 ] 23747 2.896% 93.244% # ( 10, 15 ] 4445 0.542% 93.786% ( 15, 22 ] 593 0.072% 93.859% ( 22, 34 ] 99 0.012% 93.871% ( 34, 51 ] 172 0.021% 93.892% ( 51, 76 ] 117 0.014% 93.906% ( 76, 110 ] 598 0.073% 93.979% ( 110, 170 ] 8745 1.066% 95.045% ( 170, 250 ] 16778 2.046% 97.091% ( 250, 380 ] 9390 1.145% 98.236% ( 380, 580 ] 7435 0.907% 99.143% ( 580, 870 ] 4480 0.546% 99.689% ( 870, 1300 ] 1439 0.175% 99.864% ( 1300, 1900 ] 512 0.062% 99.927% ( 1900, 2900 ] 717 0.087% 100.014% ( 2900, 4400 ] 736 0.090% 100.104% ( 4400, 6600 ] 146 0.018% 100.122% ( 6600, 9900 ] 30 0.004% 100.125% ( 9900, 14000 ] 17 0.002% 100.128% ( 14000, 22000 ] 16 0.002% 100.129% ( 22000, 33000 ] 10 0.001% 100.131% ( 33000, 50000 ] 2 0.000% 100.131% ** Level 5 read latency histogram (micros): Count: 41928 Average: 420.4038 StdDev: 736.62 Min: 90 Median: 244.9295 Max: 42143 Percentiles: P50: 244.93 P75: 443.53 P99: 3563.88 P99.9: 6489.37 P99.99: 27146.53 ------------------------------------------------------ ( 76, 110 ] 488 1.164% 1.164% ( 110, 170 ] 7398 17.645% 18.808% #### ( 170, 250 ] 13963 33.302% 52.111% ####### ( 250, 380 ] 7712 18.393% 70.504% #### ( 380, 580 ] 5934 14.153% 84.657% ### ( 580, 870 ] 3665 8.741% 93.398% ## ( 870, 1300 ] 1227 2.926% 96.325% # ( 1300, 1900 ] 393 0.937% 97.262% ( 1900, 2900 ] 503 1.200% 98.462% ( 2900, 4400 ] 510 1.216% 99.678% ( 4400, 6600 ] 98 0.234% 99.912% ( 6600, 9900 ] 18 0.043% 99.955% ( 9900, 14000 ] 6 0.014% 99.969% ( 14000, 22000 ] 6 0.014% 99.983% ( 22000, 33000 ] 6 0.014% 99.998% ( 33000, 50000 ] 2 0.005% 100.002% ** DB Stats ** Uptime(secs): 14.2 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 208.63 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.686 H:M:S, 11.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 262.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 361.648 micros/op 11002 ops/sec 20.178 seconds 221996 operations; 10.7 MB/s (54999 of 54999 found) Microseconds per read: Count: 221996 Average: 361.6599 StdDev: 859.20 Min: 1 Median: 233.0441 Max: 102260 Percentiles: P50: 233.04 P75: 359.48 P99: 2682.82 P99.9: 12260.77 P99.99: 31680.07 ------------------------------------------------------ [ 0, 1 ] 1 0.000% 0.000% ( 1, 2 ] 13 0.006% 0.006% ( 2, 3 ] 52 0.023% 0.030% ( 3, 4 ] 75 0.034% 0.064% ( 4, 6 ] 224 0.101% 0.164% ( 6, 10 ] 387 0.174% 0.339% ( 10, 15 ] 2085 0.939% 1.278% ( 15, 22 ] 3082 1.388% 2.666% ( 22, 34 ] 5057 2.278% 4.944% ( 34, 51 ] 2052 0.924% 5.869% ( 51, 76 ] 406 0.183% 6.051% ( 76, 110 ] 211 0.095% 6.147% ( 110, 170 ] 21217 9.557% 15.704% ## ( 170, 250 ] 96613 43.520% 59.224% ######### ( 250, 380 ] 41585 18.732% 77.956% #### ( 380, 580 ] 26041 11.730% 89.687% ## ( 580, 870 ] 13171 5.933% 95.620% # ( 870, 1300 ] 4941 2.226% 97.845% ( 1300, 1900 ] 1608 0.724% 98.570% ( 1900, 2900 ] 1220 0.550% 99.119% ( 2900, 4400 ] 1216 0.548% 99.667% ( 4400, 6600 ] 321 0.145% 99.812% ( 6600, 9900 ] 139 0.063% 99.874% ( 9900, 14000 ] 99 0.045% 99.919% ( 14000, 22000 ] 105 0.047% 99.966% ( 22000, 33000 ] 60 0.027% 99.993% ( 33000, 50000 ] 13 0.006% 99.999% ( 50000, 75000 ] 1 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3642108 rocksdb.bloom.filter.full.positive COUNT : 256802 rocksdb.bloom.filter.full.true.positive COUNT : 221126 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 870 rocksdb.memtable.miss COUNT : 221126 rocksdb.l0.hit COUNT : 31388 rocksdb.l1.hit COUNT : 1752 rocksdb.l2andup.hit COUNT : 187986 rocksdb.compaction.key.drop.new COUNT : 645105 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3806114 rocksdb.number.keys.read COUNT : 221996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3927909648 rocksdb.bytes.read COUNT : 221996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 500 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 4680680 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 14129326312 rocksdb.compact.write.bytes COUNT : 13458540700 rocksdb.flush.write.bytes COUNT : 3891681642 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 29 rocksdb.number.superversion_acquires COUNT : 749 rocksdb.number.superversion_releases COUNT : 725 rocksdb.number.superversion_cleanups COUNT : 184 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 168 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3865952256 rocksdb.memtable.garbage.bytes.at.flush COUNT : 6664192 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 14252821614 rocksdb.non.last.level.read.count COUNT : 3414189 rocksdb.block.checksum.compute.count COUNT : 3681780 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 232.499251 P95 : 839.090826 P99 : 2681.821165 P100 : 102259.000000 COUNT : 221996 SUM : 80111434 rocksdb.db.write.micros P50 : 1.630297 P95 : 5.191897 P99 : 8.304389 P100 : 445047.000000 COUNT : 3806114 SUM : 15222047 rocksdb.compaction.times.micros P50 : 1383333.333333 P95 : 6995000.000000 P99 : 9239000.000000 P100 : 9427694.000000 COUNT : 51 SUM : 87827507 rocksdb.compaction.times.cpu_micros P50 : 1065789.473684 P95 : 1836176.470588 P99 : 5141451.000000 P100 : 5141451.000000 COUNT : 51 SUM : 58527513 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 821.666667 P95 : 4620.000000 P99 : 8052.000000 P100 : 8938.000000 COUNT : 56 SUM : 74159 rocksdb.compaction.outfile.sync.micros P50 : 1517.241379 P95 : 6416.666667 P99 : 8958.000000 P100 : 8958.000000 COUNT : 155 SUM : 336839 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 227.931034 P95 : 1257.000000 P99 : 4270.000000 P100 : 6314.000000 COUNT : 126 SUM : 55015 rocksdb.table.open.io.micros P50 : 5166.666667 P95 : 18857.142857 P99 : 29071.428571 P100 : 32443.000000 COUNT : 500 SUM : 3158519 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.314956 P95 : 9.715083 P99 : 230.014338 P100 : 43397.000000 COUNT : 3421326 SUM : 46963575 rocksdb.read.block.get.micros P50 : 202.413352 P95 : 722.203648 P99 : 2266.975867 P100 : 42147.000000 COUNT : 259939 SUM : 72001446 rocksdb.write.raw.block.micros P50 : 0.523760 P95 : 0.995144 P99 : 3.687161 P100 : 60475.000000 COUNT : 4203097 SUM : 12753966 rocksdb.numfiles.in.singlecompaction P50 : 2.833333 P95 : 5.612500 P99 : 16.000000 P100 : 16.000000 COUNT : 71 SUM : 241 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.562359 P95 : 1118.695713 P99 : 1285.385040 P100 : 444854.000000 COUNT : 31419 SUM : 4684014 rocksdb.sst.read.micros P50 : 2.130123 P95 : 196.120717 P99 : 535.148391 P100 : 43393.000000 COUNT : 3414216 SUM : 106791192 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 221996 SUM : 221996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3806114 SUM : 3927909648 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 227435.897436 P95 : 521142.857143 P99 : 697600.000000 P100 : 789744.000000 COUNT : 56 SUM : 14427529 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 175 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0