[2024-11-27 05:34:30.817093] Starting SPDK v25.01-pre git sha1 78decfef6 / DPDK 24.03.0 initialization... [2024-11-27 05:34:30.817408] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1253688 ] Set seed to 1732682070809854 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-27 05:34:30.968489] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-27 05:34:31.079830] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-27 05:34:31.079890] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1253688' to capture a snapshot of events at runtime. [2024-11-27 05:34:31.079903] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-27 05:34:31.079924] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-27 05:34:31.079935] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1253688 for offline analysis/debug. [2024-11-27 05:34:31.081313] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 27 05:34:34 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/11/27-05:34:41 ... thread 0: (1000000,1000000) ops and (139766.1,139766.1) ops/second in (7.154810,7.154810) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 1.25 GB 43.8 0.3 0.0 0.3 1.1 0.8 0.0 1.4 36.1 129.7 8.71 4.16 11 0.792 319K 1842 0.0 0.0 L1 2/2 113.96 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 128.78 MB 1.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 29/8 1.73 GB 1.3 2.3 1.6 0.7 2.3 1.6 0.0 1.4 101.0 98.2 23.79 13.25 11 2.162 1993K 9900 0.0 0.0 L4 161/30 11.91 GB 1.0 2.8 1.7 1.1 2.7 1.6 0.1 1.6 114.8 110.1 25.12 15.01 14 1.794 560K 0 0.0 0.0 L5 112/0 8.98 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 317/49 24.12 GB 0.0 5.5 3.3 2.2 6.1 3.9 0.1 7.6 97.2 108.2 57.61 32.42 36 1.600 2873K 11K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.5 3.3 2.2 6.1 3.9 0.1 7.6 97.2 108.2 57.61 32.42 36 1.600 2873K 11K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.5 3.3 2.2 5.3 3.1 0.0 0.0 109.7 106.0 51.06 29.79 26 1.964 2873K 11K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 124.8 6.55 2.63 10 0.655 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.2 total, 7.2 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.09 GB write, 862.41 MB/s write, 5.47 GB read, 775.23 MB/s read, 57.6 seconds Interval compaction: 6.09 GB write, 870.77 MB/s write, 5.47 GB read, 782.74 MB/s read, 57.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 19, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 161404 Average: 60.0677 StdDev: 646.19 Min: 1 Median: 2.2900 Max: 66972 Percentiles: P50: 2.29 P75: 3.35 P99: 1178.82 P99.9: 5290.09 P99.99: 25340.97 ------------------------------------------------------ [ 0, 1 ] 2511 1.556% 1.556% ( 1, 2 ] 63912 39.598% 41.153% ######## ( 2, 3 ] 49242 30.509% 71.662% ###### ( 3, 4 ] 15368 9.521% 81.183% ## ( 4, 6 ] 17047 10.562% 91.745% ## ( 6, 10 ] 4555 2.822% 94.567% # ( 10, 15 ] 1060 0.657% 95.224% ( 15, 22 ] 201 0.125% 95.348% ( 22, 34 ] 29 0.018% 95.366% ( 34, 51 ] 39 0.024% 95.390% ( 51, 76 ] 15 0.009% 95.400% ( 76, 110 ] 10 0.006% 95.406% ( 110, 170 ] 48 0.030% 95.436% ( 170, 250 ] 222 0.138% 95.573% ( 250, 380 ] 806 0.499% 96.073% ( 380, 580 ] 1962 1.216% 97.288% ( 580, 870 ] 2085 1.292% 98.580% ( 870, 1300 ] 944 0.585% 99.165% ( 1300, 1900 ] 286 0.177% 99.342% ( 1900, 2900 ] 346 0.214% 99.556% ( 2900, 4400 ] 502 0.311% 99.867% ( 4400, 6600 ] 130 0.081% 99.948% ( 6600, 9900 ] 45 0.028% 99.976% ( 9900, 14000 ] 7 0.004% 99.980% ( 14000, 22000 ] 11 0.007% 99.987% ( 22000, 33000 ] 16 0.010% 99.997% ( 33000, 50000 ] 13 0.008% 100.005% ( 50000, 75000 ] 4 0.002% 100.007% ** Level 1 read latency histogram (micros): Count: 154 Average: 969.6688 StdDev: 1134.68 Min: 108 Median: 632.2000 Max: 7770 Percentiles: P50: 632.20 P75: 855.50 P99: 6303.00 P99.9: 7770.00 P99.99: 7770.00 ------------------------------------------------------ ( 76, 110 ] 1 0.649% 0.649% ( 110, 170 ] 3 1.948% 2.597% ( 170, 250 ] 3 1.948% 4.545% ( 250, 380 ] 17 11.039% 15.584% ## ( 380, 580 ] 44 28.571% 44.156% ###### ( 580, 870 ] 50 32.468% 76.623% ###### ( 870, 1300 ] 15 9.740% 86.364% ## ( 1300, 1900 ] 5 3.247% 89.610% # ( 1900, 2900 ] 4 2.597% 92.208% # ( 2900, 4400 ] 7 4.545% 96.753% # ( 4400, 6600 ] 4 2.597% 99.351% # ( 6600, 9900 ] 1 0.649% 100.000% ** Level 2 read latency histogram (micros): Count: 354629 Average: 17.4202 StdDev: 350.07 Min: 1 Median: 2.3198 Max: 49766 Percentiles: P50: 2.32 P75: 3.30 P99: 13.84 P99.9: 2155.36 P99.99: 3542.69 ------------------------------------------------------ [ 0, 1 ] 3573 1.008% 1.008% ( 1, 2 ] 137702 38.830% 39.837% ######## ( 2, 3 ] 112693 31.778% 71.615% ###### ( 3, 4 ] 39835 11.233% 82.848% ## ( 4, 6 ] 44388 12.517% 95.365% ### ( 6, 10 ] 10588 2.986% 98.350% # ( 10, 15 ] 2997 0.845% 99.195% ( 15, 22 ] 542 0.153% 99.348% ( 22, 34 ] 69 0.019% 99.368% ( 34, 51 ] 110 0.031% 99.399% ( 51, 76 ] 70 0.020% 99.419% ( 76, 110 ] 16 0.005% 99.423% ( 110, 170 ] 25 0.007% 99.430% ( 170, 250 ] 79 0.022% 99.452% ( 250, 380 ] 247 0.070% 99.522% ( 380, 580 ] 482 0.136% 99.658% ( 580, 870 ] 395 0.111% 99.769% ( 870, 1300 ] 231 0.065% 99.834% ( 1300, 1900 ] 167 0.047% 99.882% ( 1900, 2900 ] 256 0.072% 99.954% ( 2900, 4400 ] 300 0.085% 100.038% ( 4400, 6600 ] 115 0.032% 100.071% ( 6600, 9900 ] 41 0.012% 100.082% ( 9900, 14000 ] 18 0.005% 100.087% ( 14000, 22000 ] 30 0.008% 100.096% ( 22000, 33000 ] 13 0.004% 100.100% ( 33000, 50000 ] 6 0.002% 100.101% ** Level 3 read latency histogram (micros): Count: 592005 Average: 17.7791 StdDev: 330.49 Min: 1 Median: 2.3835 Max: 54080 Percentiles: P50: 2.38 P75: 3.34 P99: 14.66 P99.9: 1189.92 P99.99: 1980.00 ------------------------------------------------------ [ 0, 1 ] 3773 0.637% 0.637% ( 1, 2 ] 214531 36.238% 36.875% ####### ( 2, 3 ] 202583 34.220% 71.095% ####### ( 3, 4 ] 67999 11.486% 82.581% ## ( 4, 6 ] 75494 12.752% 95.334% ### ( 6, 10 ] 17598 2.973% 98.306% # ( 10, 15 ] 4407 0.744% 99.051% ( 15, 22 ] 706 0.119% 99.170% ( 22, 34 ] 103 0.017% 99.187% ( 34, 51 ] 217 0.037% 99.224% ( 51, 76 ] 140 0.024% 99.248% ( 76, 110 ] 39 0.007% 99.254% ( 110, 170 ] 77 0.013% 99.267% ( 170, 250 ] 183 0.031% 99.298% ( 250, 380 ] 565 0.095% 99.394% ( 380, 580 ] 1243 0.210% 99.604% ( 580, 870 ] 1258 0.212% 99.816% ( 870, 1300 ] 668 0.113% 99.929% ( 1300, 1900 ] 331 0.056% 99.985% ( 1900, 2900 ] 385 0.065% 100.050% ( 2900, 4400 ] 474 0.080% 100.130% ( 4400, 6600 ] 141 0.024% 100.154% ( 6600, 9900 ] 78 0.013% 100.167% ( 9900, 14000 ] 27 0.005% 100.171% ( 14000, 22000 ] 30 0.005% 100.177% ( 22000, 33000 ] 14 0.002% 100.179% ( 33000, 50000 ] 13 0.002% 100.181% ( 50000, 75000 ] 2 0.000% 100.181% ** Level 4 read latency histogram (micros): Count: 587096 Average: 32.4881 StdDev: 430.76 Min: 1 Median: 2.4371 Max: 56117 Percentiles: P50: 2.44 P75: 3.52 P99: 591.12 P99.9: 1742.35 P99.99: 2387.23 ------------------------------------------------------ [ 0, 1 ] 3959 0.674% 0.674% ( 1, 2 ] 203649 34.688% 35.362% ####### ( 2, 3 ] 196622 33.491% 68.852% ####### ( 3, 4 ] 69513 11.840% 80.693% ## ( 4, 6 ] 77539 13.207% 93.900% ### ( 6, 10 ] 18393 3.133% 97.033% # ( 10, 15 ] 4605 0.784% 97.817% ( 15, 22 ] 739 0.126% 97.943% ( 22, 34 ] 115 0.020% 97.963% ( 34, 51 ] 228 0.039% 98.001% ( 51, 76 ] 182 0.031% 98.032% ( 76, 110 ] 68 0.012% 98.044% ( 110, 170 ] 118 0.020% 98.064% ( 170, 250 ] 511 0.087% 98.151% ( 250, 380 ] 1527 0.260% 98.411% ( 380, 580 ] 3329 0.567% 98.978% ( 580, 870 ] 3391 0.578% 99.556% ( 870, 1300 ] 1526 0.260% 99.816% ( 1300, 1900 ] 674 0.115% 99.931% ( 1900, 2900 ] 721 0.123% 100.053% ( 2900, 4400 ] 989 0.168% 100.222% ( 4400, 6600 ] 239 0.041% 100.262% ( 6600, 9900 ] 81 0.014% 100.276% ( 9900, 14000 ] 34 0.006% 100.282% ( 14000, 22000 ] 52 0.009% 100.291% ( 22000, 33000 ] 28 0.005% 100.296% ( 33000, 50000 ] 21 0.004% 100.299% ( 50000, 75000 ] 2 0.000% 100.300% ** Level 5 read latency histogram (micros): Count: 8212 Average: 1114.7589 StdDev: 2392.04 Min: 99 Median: 646.4185 Max: 70118 Percentiles: P50: 646.42 P75: 1047.53 P99: 6246.52 P99.9: 41049.50 P99.99: 69867.50 ------------------------------------------------------ ( 76, 110 ] 1 0.012% 0.012% ( 110, 170 ] 72 0.877% 0.889% ( 170, 250 ] 337 4.104% 4.993% # ( 250, 380 ] 984 11.982% 16.975% ## ( 380, 580 ] 2226 27.107% 44.082% ##### ( 580, 870 ] 2122 25.840% 69.922% ##### ( 870, 1300 ] 1010 12.299% 82.221% ## ( 1300, 1900 ] 420 5.114% 87.336% # ( 1900, 2900 ] 384 4.676% 92.012% # ( 2900, 4400 ] 474 5.772% 97.784% # ( 4400, 6600 ] 119 1.449% 99.233% ( 6600, 9900 ] 23 0.280% 99.513% ( 9900, 14000 ] 8 0.097% 99.610% ( 14000, 22000 ] 8 0.097% 99.708% ( 22000, 33000 ] 12 0.146% 99.854% ( 33000, 50000 ] 8 0.097% 99.951% ( 50000, 75000 ] 4 0.049% 100.000% ** DB Stats ** Uptime(secs): 7.2 total, 7.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 136.21 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.232 H:M:S, 30.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 137.51 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.232 H:M:S, 31.2 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/27-05:34:46 ... thread 0: (1000000,2000000) ops and (180153.9,157410.7) ops/second in (5.550810,12.705620) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/12 1.84 GB 93.8 0.9 0.0 0.9 2.7 1.8 0.0 1.5 56.6 177.5 15.56 9.57 28 0.556 894K 4934 0.0 0.0 L1 7/0 576.12 MB 57.6 0.6 0.5 0.1 0.6 0.5 0.0 1.2 52.4 51.8 11.12 3.48 1 11.122 591K 6338 0.0 0.0 L3 15/0 934.87 MB 0.9 2.6 1.7 0.9 2.6 1.7 0.0 1.5 105.1 102.4 25.61 14.74 12 2.134 2155K 9900 0.0 0.0 L4 144/0 9.73 GB 1.0 6.5 2.3 4.2 5.8 1.6 0.4 2.5 130.9 116.4 50.80 32.33 25 2.032 1252K 0 0.0 0.0 L5 145/0 11.50 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 327/12 24.55 GB 0.0 10.6 4.5 6.1 11.6 5.5 3.0 6.3 104.8 115.2 103.09 60.11 66 1.562 4893K 21K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.6 4.5 6.1 11.6 5.5 3.0 6.3 104.8 115.2 103.09 60.11 66 1.562 4893K 21K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.6 4.5 6.1 9.8 3.7 0.0 0.0 117.4 108.6 92.01 54.30 41 2.244 4893K 21K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 170.3 11.08 5.82 25 0.443 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.8 total, 12.7 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.60 GB write, 929.46 MB/s write, 10.55 GB read, 845.78 MB/s read, 103.1 seconds Interval compaction: 11.60 GB write, 934.53 MB/s write, 10.55 GB read, 850.39 MB/s read, 103.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 25, interval: 26 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 413219 Average: 61.1432 StdDev: 485.41 Min: 0 Median: 1.9057 Max: 66972 Percentiles: P50: 1.91 P75: 3.09 P99: 847.01 P99.9: 3730.20 P99.99: 4925.85 ------------------------------------------------------ [ 0, 1 ] 33461 8.098% 8.098% ## ( 1, 2 ] 191167 46.263% 54.361% ######### ( 2, 3 ] 83100 20.110% 74.471% #### ( 3, 4 ] 23456 5.676% 80.147% # ( 4, 6 ] 22664 5.485% 85.632% # ( 6, 10 ] 6471 1.566% 87.198% ( 10, 15 ] 1539 0.372% 87.571% ( 15, 22 ] 290 0.070% 87.641% ( 22, 34 ] 47 0.011% 87.652% ( 34, 51 ] 42 0.010% 87.662% ( 51, 76 ] 17 0.004% 87.666% ( 76, 110 ] 136 0.033% 87.699% ( 110, 170 ] 5327 1.289% 88.988% ( 170, 250 ] 20838 5.043% 94.031% # ( 250, 380 ] 10333 2.501% 96.532% # ( 380, 580 ] 6074 1.470% 98.002% ( 580, 870 ] 4481 1.084% 99.086% ( 870, 1300 ] 1878 0.454% 99.541% ( 1300, 1900 ] 504 0.122% 99.663% ( 1900, 2900 ] 570 0.138% 99.801% ( 2900, 4400 ] 744 0.180% 99.981% ( 4400, 6600 ] 166 0.040% 100.021% ( 6600, 9900 ] 55 0.013% 100.034% ( 9900, 14000 ] 15 0.004% 100.038% ( 14000, 22000 ] 20 0.005% 100.043% ( 22000, 33000 ] 19 0.005% 100.047% ( 33000, 50000 ] 20 0.005% 100.052% ( 50000, 75000 ] 4 0.001% 100.053% ** Level 1 read latency histogram (micros): Count: 30165 Average: 15.3054 StdDev: 119.27 Min: 1 Median: 2.2082 Max: 7770 Percentiles: P50: 2.21 P75: 3.61 P99: 316.81 P99.9: 1238.14 P99.99: 5490.92 ------------------------------------------------------ [ 0, 1 ] 700 2.321% 2.321% ( 1, 2 ] 12966 42.984% 45.304% ######### ( 2, 3 ] 6804 22.556% 67.860% ##### ( 3, 4 ] 3553 11.779% 79.639% ## ( 4, 6 ] 3924 13.008% 92.647% ### ( 6, 10 ] 887 2.940% 95.588% # ( 10, 15 ] 225 0.746% 96.333% ( 15, 22 ] 104 0.345% 96.678% ( 22, 34 ] 14 0.046% 96.725% ( 34, 51 ] 2 0.007% 96.731% ( 76, 110 ] 5 0.017% 96.748% ( 110, 170 ] 143 0.474% 97.222% ( 170, 250 ] 431 1.429% 98.651% ( 250, 380 ] 205 0.680% 99.330% ( 380, 580 ] 84 0.278% 99.609% ( 580, 870 ] 69 0.229% 99.838% ( 870, 1300 ] 22 0.073% 99.910% ( 1300, 1900 ] 8 0.027% 99.937% ( 1900, 2900 ] 7 0.023% 99.960% ( 2900, 4400 ] 7 0.023% 99.983% ( 4400, 6600 ] 4 0.013% 99.997% ( 6600, 9900 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 387399 Average: 16.7802 StdDev: 340.26 Min: 1 Median: 2.3257 Max: 49766 Percentiles: P50: 2.33 P75: 3.30 P99: 13.75 P99.9: 2106.04 P99.99: 3582.20 ------------------------------------------------------ [ 0, 1 ] 3726 0.962% 0.962% ( 1, 2 ] 149467 38.582% 39.544% ######## ( 2, 3 ] 124351 32.099% 71.643% ###### ( 3, 4 ] 43391 11.201% 82.844% ## ( 4, 6 ] 48544 12.531% 95.374% ### ( 6, 10 ] 11625 3.001% 98.375% # ( 10, 15 ] 3226 0.833% 99.208% ( 15, 22 ] 574 0.148% 99.356% ( 22, 34 ] 76 0.020% 99.376% ( 34, 51 ] 119 0.031% 99.406% ( 51, 76 ] 79 0.020% 99.427% ( 76, 110 ] 20 0.005% 99.432% ( 110, 170 ] 31 0.008% 99.440% ( 170, 250 ] 85 0.022% 99.462% ( 250, 380 ] 271 0.070% 99.532% ( 380, 580 ] 518 0.134% 99.665% ( 580, 870 ] 424 0.109% 99.775% ( 870, 1300 ] 256 0.066% 99.841% ( 1300, 1900 ] 174 0.045% 99.886% ( 1900, 2900 ] 265 0.068% 99.954% ( 2900, 4400 ] 304 0.078% 100.033% ( 4400, 6600 ] 117 0.030% 100.063% ( 6600, 9900 ] 44 0.011% 100.074% ( 9900, 14000 ] 20 0.005% 100.080% ( 14000, 22000 ] 32 0.008% 100.088% ( 22000, 33000 ] 13 0.003% 100.091% ( 33000, 50000 ] 6 0.002% 100.093% ** Level 3 read latency histogram (micros): Count: 734732 Average: 17.7737 StdDev: 320.89 Min: 1 Median: 2.3669 Max: 54080 Percentiles: P50: 2.37 P75: 3.32 P99: 191.82 P99.9: 1180.24 P99.99: 1993.60 ------------------------------------------------------ [ 0, 1 ] 5344 0.727% 0.727% ( 1, 2 ] 270995 36.884% 37.611% ####### ( 2, 3 ] 248095 33.767% 71.378% ####### ( 3, 4 ] 82604 11.243% 82.620% ## ( 4, 6 ] 90843 12.364% 94.984% ## ( 6, 10 ] 21726 2.957% 97.941% # ( 10, 15 ] 5310 0.723% 98.664% ( 15, 22 ] 855 0.116% 98.781% ( 22, 34 ] 121 0.016% 98.797% ( 34, 51 ] 249 0.034% 98.831% ( 51, 76 ] 162 0.022% 98.853% ( 76, 110 ] 62 0.008% 98.861% ( 110, 170 ] 528 0.072% 98.933% ( 170, 250 ] 1799 0.245% 99.178% ( 250, 380 ] 1323 0.180% 99.358% ( 380, 580 ] 1740 0.237% 99.595% ( 580, 870 ] 1628 0.222% 99.817% ( 870, 1300 ] 850 0.116% 99.932% ( 1300, 1900 ] 384 0.052% 99.984% ( 1900, 2900 ] 433 0.059% 100.043% ( 2900, 4400 ] 537 0.073% 100.117% ( 4400, 6600 ] 145 0.020% 100.136% ( 6600, 9900 ] 79 0.011% 100.147% ( 9900, 14000 ] 31 0.004% 100.151% ( 14000, 22000 ] 38 0.005% 100.156% ( 22000, 33000 ] 16 0.002% 100.159% ( 33000, 50000 ] 16 0.002% 100.161% ( 50000, 75000 ] 2 0.000% 100.161% ** Level 4 read latency histogram (micros): Count: 1114302 Average: 29.0226 StdDev: 368.60 Min: 1 Median: 2.4189 Max: 57407 Percentiles: P50: 2.42 P75: 3.54 P99: 467.41 P99.9: 1152.77 P99.99: 1446.60 ------------------------------------------------------ [ 0, 1 ] 8221 0.738% 0.738% ( 1, 2 ] 396927 35.621% 36.359% ####### ( 2, 3 ] 362823 32.561% 68.919% ####### ( 3, 4 ] 126582 11.360% 80.279% ## ( 4, 6 ] 138254 12.407% 92.686% ## ( 6, 10 ] 33410 2.998% 95.685% # ( 10, 15 ] 8185 0.735% 96.419% ( 15, 22 ] 1250 0.112% 96.531% ( 22, 34 ] 167 0.015% 96.546% ( 34, 51 ] 317 0.028% 96.575% ( 51, 76 ] 277 0.025% 96.600% ( 76, 110 ] 187 0.017% 96.617% ( 110, 170 ] 3528 0.317% 96.933% ( 170, 250 ] 13162 1.181% 98.114% ( 250, 380 ] 7286 0.654% 98.768% ( 380, 580 ] 5910 0.530% 99.299% ( 580, 870 ] 5205 0.467% 99.766% ( 870, 1300 ] 2276 0.204% 99.970% ( 1300, 1900 ] 915 0.082% 100.052% ( 1900, 2900 ] 988 0.089% 100.141% ( 2900, 4400 ] 1255 0.113% 100.253% ( 4400, 6600 ] 293 0.026% 100.280% ( 6600, 9900 ] 93 0.008% 100.288% ( 9900, 14000 ] 42 0.004% 100.292% ( 14000, 22000 ] 82 0.007% 100.299% ( 22000, 33000 ] 46 0.004% 100.303% ( 33000, 50000 ] 24 0.002% 100.305% ( 50000, 75000 ] 3 0.000% 100.306% ** Level 5 read latency histogram (micros): Count: 26808 Average: 572.9085 StdDev: 1497.31 Min: 91 Median: 288.9785 Max: 70118 Percentiles: P50: 288.98 P75: 554.39 P99: 4231.66 P99.9: 23722.29 P99.99: 58245.00 ------------------------------------------------------ ( 76, 110 ] 69 0.257% 0.257% ( 110, 170 ] 2527 9.426% 9.684% ## ( 170, 250 ] 9355 34.896% 44.580% ####### ( 250, 380 ] 4846 18.077% 62.657% #### ( 380, 580 ] 3795 14.156% 76.813% ### ( 580, 870 ] 3037 11.329% 88.142% ## ( 870, 1300 ] 1403 5.234% 93.375% # ( 1300, 1900 ] 528 1.970% 95.345% ( 1900, 2900 ] 473 1.764% 97.109% ( 2900, 4400 ] 571 2.130% 99.239% ( 4400, 6600 ] 130 0.485% 99.724% ( 6600, 9900 ] 23 0.086% 99.810% ( 9900, 14000 ] 11 0.041% 99.851% ( 14000, 22000 ] 11 0.041% 99.892% ( 22000, 33000 ] 14 0.052% 99.944% ( 33000, 50000 ] 11 0.041% 99.985% ( 50000, 75000 ] 4 0.015% 100.000% ** DB Stats ** Uptime(secs): 12.8 total, 5.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 154.06 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.676 H:M:S, 28.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 177.31 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.445 H:M:S, 26.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/27-05:34:50 ... thread 0: (1000000,3000000) ops and (250138.9,179604.2) ops/second in (3.997779,16.703399) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 2.81 GB 12.6 2.5 0.0 2.5 5.3 2.8 0.0 1.9 64.3 136.1 40.09 20.07 50 0.802 2618K 14K 0.0 0.0 L1 7/7 576.12 MB 0.0 0.6 0.5 0.1 0.6 0.5 0.0 1.2 52.4 51.8 11.12 3.48 1 11.122 591K 6338 0.0 0.0 L3 15/0 934.87 MB 0.9 2.6 1.7 0.9 2.6 1.7 0.0 1.5 105.1 102.4 25.61 14.74 12 2.134 2155K 9900 0.0 0.0 L4 144/0 9.73 GB 1.0 6.5 2.3 4.2 5.8 1.6 0.4 2.5 130.9 116.4 50.80 32.33 25 2.032 1252K 0 0.0 0.0 L5 145/0 11.50 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 325/19 25.52 GB 0.0 12.2 4.5 7.7 14.2 6.5 3.0 5.0 98.0 114.2 127.62 70.61 88 1.450 6617K 30K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.2 4.5 7.7 14.2 6.5 3.0 5.0 98.0 114.2 127.62 70.61 88 1.450 6617K 30K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.2 4.5 7.7 11.4 3.7 0.0 0.0 110.2 102.9 113.43 62.08 47 2.413 6617K 30K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 203.8 14.20 8.53 41 0.346 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.8 total, 16.7 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.23 GB write, 868.61 MB/s write, 12.21 GB read, 745.43 MB/s read, 127.6 seconds Interval compaction: 14.23 GB write, 872.21 MB/s write, 12.21 GB read, 748.52 MB/s read, 127.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 25, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 25, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 807877 Average: 40.0175 StdDev: 350.50 Min: 0 Median: 1.7404 Max: 66972 Percentiles: P50: 1.74 P75: 2.66 P99: 582.58 P99.9: 2355.56 P99.99: 3739.14 ------------------------------------------------------ [ 0, 1 ] 87065 10.777% 10.777% ## ( 1, 2 ] 427992 52.977% 63.754% ########### ( 2, 3 ] 138014 17.084% 80.838% ### ( 3, 4 ] 34188 4.232% 85.070% # ( 4, 6 ] 29618 3.666% 88.736% # ( 6, 10 ] 9715 1.203% 89.938% ( 10, 15 ] 1906 0.236% 90.174% ( 15, 22 ] 342 0.042% 90.217% ( 22, 34 ] 69 0.009% 90.225% ( 34, 51 ] 44 0.005% 90.231% ( 51, 76 ] 18 0.002% 90.233% ( 76, 110 ] 339 0.042% 90.275% ( 110, 170 ] 10980 1.359% 91.634% ( 170, 250 ] 37226 4.608% 96.242% # ( 250, 380 ] 15029 1.860% 98.102% ( 380, 580 ] 7212 0.893% 98.995% ( 580, 870 ] 4642 0.575% 99.569% ( 870, 1300 ] 1902 0.235% 99.805% ( 1300, 1900 ] 508 0.063% 99.868% ( 1900, 2900 ] 571 0.071% 99.938% ( 2900, 4400 ] 744 0.092% 100.031% ( 4400, 6600 ] 166 0.021% 100.051% ( 6600, 9900 ] 56 0.007% 100.058% ( 9900, 14000 ] 15 0.002% 100.060% ( 14000, 22000 ] 20 0.002% 100.062% ( 22000, 33000 ] 19 0.002% 100.065% ( 33000, 50000 ] 20 0.002% 100.067% ( 50000, 75000 ] 4 0.000% 100.068% ** Level 1 read latency histogram (micros): Count: 32468 Average: 28.3129 StdDev: 127.03 Min: 1 Median: 2.3676 Max: 7770 Percentiles: P50: 2.37 P75: 4.09 P99: 374.73 P99.9: 1212.67 P99.99: 5364.26 ------------------------------------------------------ [ 0, 1 ] 700 2.156% 2.156% ( 1, 2 ] 13000 40.039% 42.195% ######## ( 2, 3 ] 6894 21.233% 63.429% #### ( 3, 4 ] 3580 11.026% 74.455% ## ( 4, 6 ] 3962 12.203% 86.658% ## ( 6, 10 ] 893 2.750% 89.408% # ( 10, 15 ] 226 0.696% 90.104% ( 15, 22 ] 104 0.320% 90.424% ( 22, 34 ] 14 0.043% 90.468% ( 34, 51 ] 2 0.006% 90.474% ( 76, 110 ] 39 0.120% 90.594% ( 110, 170 ] 653 2.011% 92.605% ( 170, 250 ] 1563 4.814% 97.419% # ( 250, 380 ] 535 1.648% 99.067% ( 380, 580 ] 169 0.521% 99.587% ( 580, 870 ] 84 0.259% 99.846% ( 870, 1300 ] 22 0.068% 99.914% ( 1300, 1900 ] 9 0.028% 99.941% ( 1900, 2900 ] 7 0.022% 99.963% ( 2900, 4400 ] 7 0.022% 99.985% ( 4400, 6600 ] 4 0.012% 99.997% ( 6600, 9900 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 387399 Average: 16.7802 StdDev: 340.26 Min: 1 Median: 2.3257 Max: 49766 Percentiles: P50: 2.33 P75: 3.30 P99: 13.75 P99.9: 2106.04 P99.99: 3582.20 ------------------------------------------------------ [ 0, 1 ] 3726 0.962% 0.962% ( 1, 2 ] 149467 38.582% 39.544% ######## ( 2, 3 ] 124351 32.099% 71.643% ###### ( 3, 4 ] 43391 11.201% 82.844% ## ( 4, 6 ] 48544 12.531% 95.374% ### ( 6, 10 ] 11625 3.001% 98.375% # ( 10, 15 ] 3226 0.833% 99.208% ( 15, 22 ] 574 0.148% 99.356% ( 22, 34 ] 76 0.020% 99.376% ( 34, 51 ] 119 0.031% 99.406% ( 51, 76 ] 79 0.020% 99.427% ( 76, 110 ] 20 0.005% 99.432% ( 110, 170 ] 31 0.008% 99.440% ( 170, 250 ] 85 0.022% 99.462% ( 250, 380 ] 271 0.070% 99.532% ( 380, 580 ] 518 0.134% 99.665% ( 580, 870 ] 424 0.109% 99.775% ( 870, 1300 ] 256 0.066% 99.841% ( 1300, 1900 ] 174 0.045% 99.886% ( 1900, 2900 ] 265 0.068% 99.954% ( 2900, 4400 ] 304 0.078% 100.033% ( 4400, 6600 ] 117 0.030% 100.063% ( 6600, 9900 ] 44 0.011% 100.074% ( 9900, 14000 ] 20 0.005% 100.080% ( 14000, 22000 ] 32 0.008% 100.088% ( 22000, 33000 ] 13 0.003% 100.091% ( 33000, 50000 ] 6 0.002% 100.093% ** Level 3 read latency histogram (micros): Count: 738460 Average: 18.7732 StdDev: 320.45 Min: 1 Median: 2.3744 Max: 54080 Percentiles: P50: 2.37 P75: 3.36 P99: 237.11 P99.9: 1179.79 P99.99: 1992.73 ------------------------------------------------------ [ 0, 1 ] 5344 0.724% 0.724% ( 1, 2 ] 271000 36.698% 37.422% ####### ( 2, 3 ] 248104 33.597% 71.019% ####### ( 3, 4 ] 82605 11.186% 82.205% ## ( 4, 6 ] 90843 12.302% 94.507% ## ( 6, 10 ] 21726 2.942% 97.449% # ( 10, 15 ] 5310 0.719% 98.168% ( 15, 22 ] 855 0.116% 98.284% ( 22, 34 ] 121 0.016% 98.300% ( 34, 51 ] 249 0.034% 98.334% ( 51, 76 ] 162 0.022% 98.356% ( 76, 110 ] 95 0.013% 98.369% ( 110, 170 ] 1390 0.188% 98.557% ( 170, 250 ] 3900 0.528% 99.085% ( 250, 380 ] 1878 0.254% 99.339% ( 380, 580 ] 1876 0.254% 99.593% ( 580, 870 ] 1649 0.223% 99.817% ( 870, 1300 ] 853 0.116% 99.932% ( 1300, 1900 ] 386 0.052% 99.985% ( 1900, 2900 ] 433 0.059% 100.043% ( 2900, 4400 ] 537 0.073% 100.116% ( 4400, 6600 ] 145 0.020% 100.136% ( 6600, 9900 ] 79 0.011% 100.146% ( 9900, 14000 ] 31 0.004% 100.150% ( 14000, 22000 ] 38 0.005% 100.156% ( 22000, 33000 ] 16 0.002% 100.158% ( 33000, 50000 ] 16 0.002% 100.160% ( 50000, 75000 ] 2 0.000% 100.160% ** Level 4 read latency histogram (micros): Count: 1143064 Average: 33.7321 StdDev: 365.52 Min: 1 Median: 2.4583 Max: 57407 Percentiles: P50: 2.46 P75: 3.70 P99: 482.55 P99.9: 1149.46 P99.99: 1448.49 ------------------------------------------------------ [ 0, 1 ] 8221 0.719% 0.719% ( 1, 2 ] 396966 34.728% 35.447% ####### ( 2, 3 ] 362929 31.751% 67.198% ###### ( 3, 4 ] 126613 11.077% 78.275% ## ( 4, 6 ] 138296 12.099% 90.373% ## ( 6, 10 ] 33411 2.923% 93.296% # ( 10, 15 ] 8186 0.716% 94.012% ( 15, 22 ] 1250 0.109% 94.122% ( 22, 34 ] 167 0.015% 94.136% ( 34, 51 ] 317 0.028% 94.164% ( 51, 76 ] 277 0.024% 94.188% ( 76, 110 ] 516 0.045% 94.233% ( 110, 170 ] 10007 0.875% 95.109% ( 170, 250 ] 29284 2.562% 97.671% # ( 250, 380 ] 11583 1.013% 98.684% ( 380, 580 ] 7041 0.616% 99.300% ( 580, 870 ] 5368 0.470% 99.770% ( 870, 1300 ] 2291 0.200% 99.970% ( 1300, 1900 ] 916 0.080% 100.050% ( 1900, 2900 ] 988 0.086% 100.137% ( 2900, 4400 ] 1257 0.110% 100.247% ( 4400, 6600 ] 295 0.026% 100.273% ( 6600, 9900 ] 94 0.008% 100.281% ( 9900, 14000 ] 42 0.004% 100.284% ( 14000, 22000 ] 82 0.007% 100.292% ( 22000, 33000 ] 46 0.004% 100.296% ( 33000, 50000 ] 24 0.002% 100.298% ( 50000, 75000 ] 3 0.000% 100.298% ** Level 5 read latency histogram (micros): Count: 46378 Average: 422.8850 StdDev: 1153.38 Min: 91 Median: 232.4172 Max: 70118 Percentiles: P50: 232.42 P75: 368.50 P99: 3724.00 P99.9: 11995.47 P99.99: 52263.75 ------------------------------------------------------ ( 76, 110 ] 277 0.597% 0.597% ( 110, 170 ] 7044 15.188% 15.786% ### ( 170, 250 ] 20338 43.853% 59.638% ######### ( 250, 380 ] 7816 16.853% 76.491% ### ( 380, 580 ] 4558 9.828% 86.319% ## ( 580, 870 ] 3146 6.783% 93.102% # ( 870, 1300 ] 1417 3.055% 96.158% # ( 1300, 1900 ] 531 1.145% 97.303% ( 1900, 2900 ] 473 1.020% 98.322% ( 2900, 4400 ] 572 1.233% 99.556% ( 4400, 6600 ] 131 0.282% 99.838% ( 6600, 9900 ] 23 0.050% 99.888% ( 9900, 14000 ] 11 0.024% 99.912% ( 14000, 22000 ] 11 0.024% 99.935% ( 22000, 33000 ] 14 0.030% 99.966% ( 33000, 50000 ] 11 0.024% 99.989% ( 50000, 75000 ] 4 0.009% 99.998% ** DB Stats ** Uptime(secs): 16.8 total, 4.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 176.02 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.928 H:M:S, 23.4 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 246.18 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.252 H:M:S, 6.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 413.379 micros/op 9654 ops/sec 20.198 seconds 194996 operations; 9.4 MB/s (48999 of 48999 found) Microseconds per read: Count: 194996 Average: 413.3942 StdDev: 1272.07 Min: 2 Median: 243.0892 Max: 86695 Percentiles: P50: 243.09 P75: 373.25 P99: 3493.46 P99.9: 19571.71 P99.99: 52083.89 ------------------------------------------------------ ( 1, 2 ] 16 0.008% 0.008% ( 2, 3 ] 76 0.039% 0.047% ( 3, 4 ] 104 0.053% 0.101% ( 4, 6 ] 216 0.111% 0.211% ( 6, 10 ] 249 0.128% 0.339% ( 10, 15 ] 2157 1.106% 1.445% ( 15, 22 ] 4109 2.107% 3.552% ( 22, 34 ] 2523 1.294% 4.846% ( 34, 51 ] 1496 0.767% 5.613% ( 51, 76 ] 259 0.133% 5.746% ( 76, 110 ] 104 0.053% 5.800% ( 110, 170 ] 13627 6.988% 12.788% # ( 170, 250 ] 79423 40.731% 53.519% ######## ( 250, 380 ] 44183 22.658% 76.177% ##### ( 380, 580 ] 23641 12.124% 88.301% ## ( 580, 870 ] 12307 6.311% 94.612% # ( 870, 1300 ] 4957 2.542% 97.154% # ( 1300, 1900 ] 1820 0.933% 98.088% ( 1900, 2900 ] 1184 0.607% 98.695% ( 2900, 4400 ] 1504 0.771% 99.466% ( 4400, 6600 ] 463 0.237% 99.704% ( 6600, 9900 ] 193 0.099% 99.803% ( 9900, 14000 ] 112 0.057% 99.860% ( 14000, 22000 ] 112 0.057% 99.917% ( 22000, 33000 ] 99 0.051% 99.968% ( 33000, 50000 ] 41 0.021% 99.989% ( 50000, 75000 ] 18 0.009% 99.998% ( 75000, 110000 ] 3 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3190750 rocksdb.bloom.filter.full.positive COUNT : 225455 rocksdb.bloom.filter.full.true.positive COUNT : 194110 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 886 rocksdb.memtable.miss COUNT : 194110 rocksdb.l0.hit COUNT : 21261 rocksdb.l1.hit COUNT : 3890 rocksdb.l2andup.hit COUNT : 168959 rocksdb.compaction.key.drop.new COUNT : 849740 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3769821 rocksdb.number.keys.read COUNT : 194996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3890455272 rocksdb.bytes.read COUNT : 194996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 505 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 4485516 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 13966858468 rocksdb.compact.write.bytes COUNT : 13088575720 rocksdb.flush.write.bytes COUNT : 3825194348 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 23 rocksdb.number.superversion_acquires COUNT : 708 rocksdb.number.superversion_releases COUNT : 680 rocksdb.number.superversion_cleanups COUNT : 172 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 167 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3800451072 rocksdb.memtable.garbage.bytes.at.flush COUNT : 7071744 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 14366630185 rocksdb.non.last.level.read.count COUNT : 3442166 rocksdb.block.checksum.compute.count COUNT : 3612104 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 242.475992 P95 : 934.586904 P99 : 3492.854291 P100 : 86694.000000 COUNT : 194996 SUM : 80462259 rocksdb.db.write.micros P50 : 1.589712 P95 : 5.171869 P99 : 7.956097 P100 : 665464.000000 COUNT : 3769821 SUM : 15002940 rocksdb.compaction.times.micros P50 : 1802777.777778 P95 : 8315000.000000 P99 : 11121574.000000 P100 : 11121574.000000 COUNT : 49 SUM : 115407624 rocksdb.compaction.times.cpu_micros P50 : 1175714.285714 P95 : 2175000.000000 P99 : 3478614.000000 P100 : 3478614.000000 COUNT : 49 SUM : 64034070 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1120.833333 P95 : 5885.000000 P99 : 7648.000000 P100 : 7648.000000 COUNT : 53 SUM : 84042 rocksdb.compaction.outfile.sync.micros P50 : 2316.666667 P95 : 8462.142857 P99 : 17040.000000 P100 : 37932.000000 COUNT : 162 SUM : 505781 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 230.000000 P95 : 2025.000000 P99 : 4325.000000 P100 : 24988.000000 COUNT : 115 SUM : 73480 rocksdb.table.open.io.micros P50 : 5775.000000 P95 : 17435.897436 P99 : 21579.487179 P100 : 78133.000000 COUNT : 505 SUM : 3331267 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.616952 P95 : 10.671946 P99 : 283.826980 P100 : 66979.000000 COUNT : 3383270 SUM : 67611198 rocksdb.read.block.get.micros P50 : 212.430515 P95 : 803.181080 P99 : 3033.788396 P100 : 70124.000000 COUNT : 228320 SUM : 72992583 rocksdb.write.raw.block.micros P50 : 0.543145 P95 : 1.535782 P99 : 3.909682 P100 : 71445.000000 COUNT : 4097821 SUM : 16254626 rocksdb.numfiles.in.singlecompaction P50 : 3.152174 P95 : 5.770000 P99 : 17.590000 P100 : 19.000000 COUNT : 63 SUM : 236 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.558967 P95 : 1104.858533 P99 : 1279.253144 P100 : 665194.000000 COUNT : 13546 SUM : 4486703 rocksdb.sst.read.micros P50 : 2.210320 P95 : 205.713604 P99 : 564.220577 P100 : 70118.000000 COUNT : 3442232 SUM : 127171075 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 194996 SUM : 194996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3769821 SUM : 3890455272 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 228888.888889 P95 : 797166.666667 P99 : 1021632.000000 P100 : 1021632.000000 COUNT : 53 SUM : 16646346 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 174 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0