[2024-12-09 13:53:40.059013] Starting SPDK v25.01-pre git sha1 b4f857a04 / DPDK 24.03.0 initialization... [2024-12-09 13:53:40.059499] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid314166 ] Set seed to 1733748820051623 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-09 13:53:40.221474] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-09 13:53:40.325396] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-09 13:53:40.325457] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 314166' to capture a snapshot of events at runtime. [2024-12-09 13:53:40.325469] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-09 13:53:40.325482] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-09 13:53:40.325492] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid314166 for offline analysis/debug. [2024-12-09 13:53:40.326734] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 9 13:53:43 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/12/09-13:53:49 ... thread 0: (1000000,1000000) ops and (149959.9,149959.9) ops/second in (6.668449,6.668449) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/8 817.23 MB 18.9 0.0 0.0 0.0 0.8 0.8 0.0 1.0 0.0 135.1 6.05 2.47 10 0.605 0 0 0.0 0.0 L1 2/0 119.74 MB 12.0 0.3 0.2 0.1 0.3 0.2 0.0 1.3 102.1 101.5 3.05 1.63 1 3.046 315K 1908 0.0 0.0 L2 1/0 104.54 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 37/8 1.94 GB 1.6 2.7 1.8 0.9 2.6 1.7 0.1 1.4 106.9 102.9 26.08 14.52 13 2.006 2031K 11K 0.0 0.0 L4 166/27 12.00 GB 1.0 2.8 1.7 1.1 2.7 1.6 0.8 1.6 122.8 117.8 23.71 14.23 14 1.694 648K 0 0.0 0.0 L5 116/0 9.10 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 332/43 24.06 GB 0.0 5.9 3.8 2.1 6.4 4.3 1.0 8.1 102.1 112.1 58.88 32.84 38 1.550 2996K 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.9 3.8 2.1 6.4 4.3 1.0 8.1 102.1 112.1 58.88 32.84 38 1.550 2996K 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.9 3.8 2.1 5.6 3.5 0.0 0.0 113.8 109.5 52.83 30.37 28 1.887 2996K 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 135.1 6.05 2.47 10 0.605 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.7 total, 6.7 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.45 GB write, 979.17 MB/s write, 5.87 GB read, 891.72 MB/s read, 58.9 seconds Interval compaction: 6.45 GB write, 989.51 MB/s write, 5.87 GB read, 901.14 MB/s read, 58.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 19, interval: 19 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 116280 Average: 49.6821 StdDev: 475.04 Min: 1 Median: 2.2013 Max: 50575 Percentiles: P50: 2.20 P75: 3.36 P99: 1136.06 P99.9: 4311.83 P99.99: 14496.00 ------------------------------------------------------ [ 0, 1 ] 2559 2.201% 2.201% ( 1, 2 ] 49321 42.416% 44.616% ######## ( 2, 3 ] 31096 26.742% 71.359% ##### ( 3, 4 ] 11795 10.144% 81.502% ## ( 4, 6 ] 12559 10.801% 92.303% ## ( 6, 10 ] 3328 2.862% 95.165% # ( 10, 15 ] 792 0.681% 95.846% ( 15, 22 ] 106 0.091% 95.937% ( 22, 34 ] 17 0.015% 95.952% ( 34, 51 ] 16 0.014% 95.966% ( 51, 76 ] 13 0.011% 95.977% ( 76, 110 ] 4 0.003% 95.980% ( 110, 170 ] 24 0.021% 96.001% ( 170, 250 ] 156 0.134% 96.135% ( 250, 380 ] 507 0.436% 96.571% ( 380, 580 ] 1192 1.025% 97.596% ( 580, 870 ] 1270 1.092% 98.689% ( 870, 1300 ] 587 0.505% 99.193% ( 1300, 1900 ] 250 0.215% 99.408% ( 1900, 2900 ] 248 0.213% 99.622% ( 2900, 4400 ] 345 0.297% 99.918% ( 4400, 6600 ] 62 0.053% 99.972% ( 6600, 9900 ] 21 0.018% 99.990% ( 9900, 14000 ] 1 0.001% 99.991% ( 14000, 22000 ] 6 0.005% 99.996% ( 22000, 33000 ] 7 0.006% 100.002% ( 33000, 50000 ] 3 0.003% 100.004% ( 50000, 75000 ] 1 0.001% 100.005% ** Level 1 read latency histogram (micros): Count: 39178 Average: 24.9895 StdDev: 256.79 Min: 1 Median: 2.2913 Max: 15531 Percentiles: P50: 2.29 P75: 3.33 P99: 602.29 P99.9: 4089.50 P99.99: 9108.91 ------------------------------------------------------ [ 0, 1 ] 399 1.018% 1.018% ( 1, 2 ] 15756 40.216% 41.235% ######## ( 2, 3 ] 11787 30.086% 71.321% ###### ( 3, 4 ] 4340 11.078% 82.398% ## ( 4, 6 ] 4661 11.897% 94.295% ## ( 6, 10 ] 1171 2.989% 97.284% # ( 10, 15 ] 250 0.638% 97.922% ( 15, 22 ] 27 0.069% 97.991% ( 22, 34 ] 7 0.018% 98.009% ( 34, 51 ] 9 0.023% 98.032% ( 51, 76 ] 10 0.026% 98.058% ( 76, 110 ] 3 0.008% 98.065% ( 110, 170 ] 10 0.026% 98.091% ( 170, 250 ] 25 0.064% 98.155% ( 250, 380 ] 103 0.263% 98.417% ( 380, 580 ] 215 0.549% 98.966% ( 580, 870 ] 172 0.439% 99.405% ( 870, 1300 ] 90 0.230% 99.635% ( 1300, 1900 ] 35 0.089% 99.724% ( 1900, 2900 ] 26 0.066% 99.791% ( 2900, 4400 ] 54 0.138% 99.929% ( 4400, 6600 ] 18 0.046% 99.974% ( 6600, 9900 ] 8 0.020% 99.995% ( 9900, 14000 ] 1 0.003% 99.997% ( 14000, 22000 ] 1 0.003% 100.000% ** Level 2 read latency histogram (micros): Count: 399473 Average: 15.8843 StdDev: 314.28 Min: 1 Median: 2.2728 Max: 45056 Percentiles: P50: 2.27 P75: 3.33 P99: 13.15 P99.9: 1197.62 P99.99: 1982.33 ------------------------------------------------------ [ 0, 1 ] 3917 0.981% 0.981% ( 1, 2 ] 163997 41.053% 42.034% ######## ( 2, 3 ] 116637 29.198% 71.232% ###### ( 3, 4 ] 44950 11.252% 82.484% ## ( 4, 6 ] 51028 12.774% 95.258% ### ( 6, 10 ] 12764 3.195% 98.453% # ( 10, 15 ] 3464 0.867% 99.320% ( 15, 22 ] 598 0.150% 99.470% ( 22, 34 ] 85 0.021% 99.491% ( 34, 51 ] 87 0.022% 99.513% ( 51, 76 ] 76 0.019% 99.532% ( 76, 110 ] 21 0.005% 99.537% ( 110, 170 ] 22 0.006% 99.543% ( 170, 250 ] 56 0.014% 99.557% ( 250, 380 ] 197 0.049% 99.606% ( 380, 580 ] 433 0.108% 99.714% ( 580, 870 ] 484 0.121% 99.836% ( 870, 1300 ] 338 0.085% 99.920% ( 1300, 1900 ] 256 0.064% 99.984% ( 1900, 2900 ] 280 0.070% 100.054% ( 2900, 4400 ] 291 0.073% 100.127% ( 4400, 6600 ] 112 0.028% 100.155% ( 6600, 9900 ] 30 0.008% 100.163% ( 9900, 14000 ] 15 0.004% 100.166% ( 14000, 22000 ] 17 0.004% 100.171% ( 22000, 33000 ] 20 0.005% 100.176% ( 33000, 50000 ] 5 0.001% 100.177% ** Level 3 read latency histogram (micros): Count: 693606 Average: 16.5679 StdDev: 262.68 Min: 1 Median: 2.3195 Max: 51076 Percentiles: P50: 2.32 P75: 3.34 P99: 14.37 P99.9: 1080.76 P99.99: 1600.83 ------------------------------------------------------ [ 0, 1 ] 5464 0.788% 0.788% ( 1, 2 ] 272365 39.268% 40.056% ######## ( 2, 3 ] 215876 31.124% 71.179% ###### ( 3, 4 ] 78239 11.280% 82.459% ## ( 4, 6 ] 88305 12.731% 95.191% ### ( 6, 10 ] 22112 3.188% 98.379% # ( 10, 15 ] 4932 0.711% 99.090% ( 15, 22 ] 676 0.097% 99.187% ( 22, 34 ] 129 0.019% 99.206% ( 34, 51 ] 209 0.030% 99.236% ( 51, 76 ] 161 0.023% 99.259% ( 76, 110 ] 53 0.008% 99.267% ( 110, 170 ] 71 0.010% 99.277% ( 170, 250 ] 212 0.031% 99.308% ( 250, 380 ] 696 0.100% 99.408% ( 380, 580 ] 1527 0.220% 99.628% ( 580, 870 ] 1507 0.217% 99.845% ( 870, 1300 ] 772 0.111% 99.957% ( 1300, 1900 ] 460 0.066% 100.023% ( 1900, 2900 ] 530 0.076% 100.099% ( 2900, 4400 ] 638 0.092% 100.191% ( 4400, 6600 ] 157 0.023% 100.214% ( 6600, 9900 ] 40 0.006% 100.220% ( 9900, 14000 ] 16 0.002% 100.222% ( 14000, 22000 ] 21 0.003% 100.225% ( 22000, 33000 ] 11 0.002% 100.227% ( 33000, 50000 ] 5 0.001% 100.228% ( 50000, 75000 ] 1 0.000% 100.228% ** Level 4 read latency histogram (micros): Count: 470741 Average: 34.6981 StdDev: 384.76 Min: 1 Median: 2.3354 Max: 52496 Percentiles: P50: 2.34 P75: 3.41 P99: 711.81 P99.9: 2567.39 P99.99: 3137.95 ------------------------------------------------------ [ 0, 1 ] 3945 0.838% 0.838% ( 1, 2 ] 182975 38.870% 39.708% ######## ( 2, 3 ] 144441 30.684% 70.391% ###### ( 3, 4 ] 52803 11.217% 81.608% ## ( 4, 6 ] 58080 12.338% 93.946% ## ( 6, 10 ] 13992 2.972% 96.919% # ( 10, 15 ] 3089 0.656% 97.575% ( 15, 22 ] 401 0.085% 97.660% ( 22, 34 ] 71 0.015% 97.675% ( 34, 51 ] 87 0.018% 97.694% ( 51, 76 ] 94 0.020% 97.714% ( 76, 110 ] 44 0.009% 97.723% ( 110, 170 ] 82 0.017% 97.740% ( 170, 250 ] 356 0.076% 97.816% ( 250, 380 ] 1296 0.275% 98.091% ( 380, 580 ] 2933 0.623% 98.714% ( 580, 870 ] 2967 0.630% 99.345% ( 870, 1300 ] 1411 0.300% 99.644% ( 1300, 1900 ] 664 0.141% 99.785% ( 1900, 2900 ] 814 0.173% 99.958% ( 2900, 4400 ] 964 0.205% 100.163% ( 4400, 6600 ] 235 0.050% 100.213% ( 6600, 9900 ] 72 0.015% 100.228% ( 9900, 14000 ] 10 0.002% 100.230% ( 14000, 22000 ] 18 0.004% 100.234% ( 22000, 33000 ] 13 0.003% 100.237% ( 33000, 50000 ] 10 0.002% 100.239% ( 50000, 75000 ] 2 0.000% 100.240% ** Level 5 read latency histogram (micros): Count: 7895 Average: 1146.2182 StdDev: 2092.51 Min: 113 Median: 674.7478 Max: 75022 Percentiles: P50: 674.75 P75: 1111.17 P99: 6206.79 P99.9: 33297.50 P99.99: 75022.00 ------------------------------------------------------ ( 110, 170 ] 45 0.570% 0.570% ( 170, 250 ] 261 3.306% 3.876% # ( 250, 380 ] 821 10.399% 14.275% ## ( 380, 580 ] 2121 26.865% 41.140% ##### ( 580, 870 ] 2141 27.118% 68.258% ##### ( 870, 1300 ] 949 12.020% 80.279% ## ( 1300, 1900 ] 390 4.940% 85.218% # ( 1900, 2900 ] 430 5.446% 90.665% # ( 2900, 4400 ] 548 6.941% 97.606% # ( 4400, 6600 ] 134 1.697% 99.303% ( 6600, 9900 ] 31 0.393% 99.696% ( 9900, 14000 ] 4 0.051% 99.747% ( 14000, 22000 ] 6 0.076% 99.823% ( 22000, 33000 ] 6 0.076% 99.899% ( 33000, 50000 ] 6 0.076% 99.975% ( 50000, 75000 ] 1 0.013% 99.987% ( 75000, 110000 ] 1 0.013% 100.000% ** DB Stats ** Uptime(secs): 6.7 total, 6.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 145.98 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.969 H:M:S, 29.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 147.51 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.969 H:M:S, 29.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/09-13:53:55 ... thread 0: (1000000,2000000) ops and (177448.4,162550.2) ops/second in (5.635441,12.303890) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/12 1.84 GB 100.1 0.9 0.0 0.9 2.7 1.8 0.0 1.5 47.7 149.5 18.47 9.43 28 0.660 893K 4897 0.0 0.0 L1 2/0 119.74 MB 12.0 0.3 0.2 0.1 0.3 0.2 0.0 1.3 102.1 101.5 3.05 1.63 1 3.046 315K 1908 0.0 0.0 L3 17/0 984.25 MB 1.0 2.8 1.9 0.9 2.7 1.8 0.1 1.4 108.3 104.3 26.72 15.04 14 1.909 2137K 11K 0.0 0.0 L4 148/0 9.57 GB 1.0 8.6 2.7 5.9 7.5 1.6 0.9 2.8 139.6 121.7 62.97 39.24 34 1.852 2078K 86K 0.0 0.0 L5 154/0 11.63 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 337/12 24.12 GB 0.0 12.6 4.9 7.7 13.2 5.5 3.5 7.2 115.8 121.6 111.21 65.35 77 1.444 5426K 104K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.6 4.9 7.7 13.2 5.5 3.5 7.2 115.8 121.6 111.21 65.35 77 1.444 5426K 104K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.6 4.9 7.7 11.4 3.6 0.0 0.0 129.0 116.5 99.85 59.74 52 1.920 5426K 104K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 166.1 11.36 5.61 25 0.454 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.4 total, 12.3 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.21 GB write, 1092.59 MB/s write, 12.57 GB read, 1040.36 MB/s read, 111.2 seconds Interval compaction: 13.21 GB write, 1098.85 MB/s write, 12.57 GB read, 1046.32 MB/s read, 111.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 27, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 375200 Average: 49.0709 StdDev: 403.32 Min: 1 Median: 1.9044 Max: 52990 Percentiles: P50: 1.90 P75: 2.93 P99: 863.26 P99.9: 3295.32 P99.99: 4013.79 ------------------------------------------------------ [ 0, 1 ] 19947 5.316% 5.316% # ( 1, 2 ] 185378 49.408% 54.724% ########## ( 2, 3 ] 82233 21.917% 76.641% #### ( 3, 4 ] 27695 7.381% 84.023% # ( 4, 6 ] 25426 6.777% 90.799% # ( 6, 10 ] 7322 1.951% 92.751% ( 10, 15 ] 1554 0.414% 93.165% ( 15, 22 ] 198 0.053% 93.218% ( 22, 34 ] 32 0.009% 93.226% ( 34, 51 ] 26 0.007% 93.233% ( 51, 76 ] 19 0.005% 93.238% ( 76, 110 ] 39 0.010% 93.249% ( 110, 170 ] 1390 0.370% 93.619% ( 170, 250 ] 5569 1.484% 95.103% ( 250, 380 ] 5303 1.413% 96.517% ( 380, 580 ] 5074 1.352% 97.869% ( 580, 870 ] 4344 1.158% 99.027% ( 870, 1300 ] 1911 0.509% 99.536% ( 1300, 1900 ] 582 0.155% 99.691% ( 1900, 2900 ] 597 0.159% 99.850% ( 2900, 4400 ] 705 0.188% 100.038% ( 4400, 6600 ] 116 0.031% 100.069% ( 6600, 9900 ] 39 0.010% 100.080% ( 9900, 14000 ] 16 0.004% 100.084% ( 14000, 22000 ] 22 0.006% 100.090% ( 22000, 33000 ] 14 0.004% 100.094% ( 33000, 50000 ] 6 0.002% 100.095% ( 50000, 75000 ] 2 0.001% 100.096% ** Level 1 read latency histogram (micros): Count: 69594 Average: 22.2478 StdDev: 283.21 Min: 1 Median: 2.2646 Max: 43249 Percentiles: P50: 2.26 P75: 3.25 P99: 549.01 P99.9: 3591.05 P99.99: 9091.75 ------------------------------------------------------ [ 0, 1 ] 536 0.770% 0.770% ( 1, 2 ] 28654 41.173% 41.943% ######## ( 2, 3 ] 21190 30.448% 72.391% ###### ( 3, 4 ] 7345 10.554% 82.945% ## ( 4, 6 ] 7926 11.389% 94.334% ## ( 6, 10 ] 2082 2.992% 97.326% # ( 10, 15 ] 385 0.553% 97.879% ( 15, 22 ] 38 0.055% 97.934% ( 22, 34 ] 10 0.014% 97.948% ( 34, 51 ] 10 0.014% 97.962% ( 51, 76 ] 12 0.017% 97.980% ( 76, 110 ] 3 0.004% 97.984% ( 110, 170 ] 44 0.063% 98.047% ( 170, 250 ] 153 0.220% 98.267% ( 250, 380 ] 205 0.295% 98.562% ( 380, 580 ] 361 0.519% 99.080% ( 580, 870 ] 308 0.443% 99.523% ( 870, 1300 ] 141 0.203% 99.726% ( 1300, 1900 ] 50 0.072% 99.797% ( 1900, 2900 ] 41 0.059% 99.856% ( 2900, 4400 ] 66 0.095% 99.951% ( 4400, 6600 ] 21 0.030% 99.981% ( 6600, 9900 ] 8 0.011% 99.993% ( 9900, 14000 ] 1 0.001% 99.994% ( 14000, 22000 ] 3 0.004% 99.999% ( 33000, 50000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 399473 Average: 15.8843 StdDev: 314.28 Min: 1 Median: 2.2728 Max: 45056 Percentiles: P50: 2.27 P75: 3.33 P99: 13.15 P99.9: 1197.62 P99.99: 1982.33 ------------------------------------------------------ [ 0, 1 ] 3917 0.981% 0.981% ( 1, 2 ] 163997 41.053% 42.034% ######## ( 2, 3 ] 116637 29.198% 71.232% ###### ( 3, 4 ] 44950 11.252% 82.484% ## ( 4, 6 ] 51028 12.774% 95.258% ### ( 6, 10 ] 12764 3.195% 98.453% # ( 10, 15 ] 3464 0.867% 99.320% ( 15, 22 ] 598 0.150% 99.470% ( 22, 34 ] 85 0.021% 99.491% ( 34, 51 ] 87 0.022% 99.513% ( 51, 76 ] 76 0.019% 99.532% ( 76, 110 ] 21 0.005% 99.537% ( 110, 170 ] 22 0.006% 99.543% ( 170, 250 ] 56 0.014% 99.557% ( 250, 380 ] 197 0.049% 99.606% ( 380, 580 ] 433 0.108% 99.714% ( 580, 870 ] 484 0.121% 99.836% ( 870, 1300 ] 338 0.085% 99.920% ( 1300, 1900 ] 256 0.064% 99.984% ( 1900, 2900 ] 280 0.070% 100.054% ( 2900, 4400 ] 291 0.073% 100.127% ( 4400, 6600 ] 112 0.028% 100.155% ( 6600, 9900 ] 30 0.008% 100.163% ( 9900, 14000 ] 15 0.004% 100.166% ( 14000, 22000 ] 17 0.004% 100.171% ( 22000, 33000 ] 20 0.005% 100.176% ( 33000, 50000 ] 5 0.001% 100.177% ** Level 3 read latency histogram (micros): Count: 906776 Average: 17.3883 StdDev: 265.45 Min: 1 Median: 2.3244 Max: 51076 Percentiles: P50: 2.32 P75: 3.36 P99: 133.88 P99.9: 1200.74 P99.99: 1841.74 ------------------------------------------------------ [ 0, 1 ] 7651 0.844% 0.844% ( 1, 2 ] 354924 39.141% 39.985% ######## ( 2, 3 ] 279925 30.870% 70.855% ###### ( 3, 4 ] 102997 11.359% 82.214% ## ( 4, 6 ] 115005 12.683% 94.897% ### ( 6, 10 ] 28990 3.197% 98.094% # ( 10, 15 ] 6535 0.721% 98.815% ( 15, 22 ] 902 0.099% 98.914% ( 22, 34 ] 155 0.017% 98.931% ( 34, 51 ] 258 0.028% 98.960% ( 51, 76 ] 191 0.021% 98.981% ( 76, 110 ] 65 0.007% 98.988% ( 110, 170 ] 277 0.031% 99.018% ( 170, 250 ] 923 0.102% 99.120% ( 250, 380 ] 1380 0.152% 99.272% ( 380, 580 ] 2361 0.260% 99.533% ( 580, 870 ] 2398 0.264% 99.797% ( 870, 1300 ] 1212 0.134% 99.931% ( 1300, 1900 ] 594 0.066% 99.996% ( 1900, 2900 ] 661 0.073% 100.069% ( 2900, 4400 ] 801 0.088% 100.158% ( 4400, 6600 ] 174 0.019% 100.177% ( 6600, 9900 ] 52 0.006% 100.183% ( 9900, 14000 ] 21 0.002% 100.185% ( 14000, 22000 ] 29 0.003% 100.188% ( 22000, 33000 ] 17 0.002% 100.190% ( 33000, 50000 ] 7 0.001% 100.191% ( 50000, 75000 ] 1 0.000% 100.191% ** Level 4 read latency histogram (micros): Count: 1496939 Average: 23.3125 StdDev: 300.49 Min: 1 Median: 2.3332 Max: 53019 Percentiles: P50: 2.33 P75: 3.42 P99: 388.16 P99.9: 1058.53 P99.99: 1258.37 ------------------------------------------------------ [ 0, 1 ] 14116 0.943% 0.943% ( 1, 2 ] 584677 39.058% 40.001% ######## ( 2, 3 ] 449235 30.010% 70.011% ###### ( 3, 4 ] 175768 11.742% 81.753% ## ( 4, 6 ] 186339 12.448% 94.201% ## ( 6, 10 ] 46558 3.110% 97.311% # ( 10, 15 ] 11119 0.743% 98.054% ( 15, 22 ] 1346 0.090% 98.144% ( 22, 34 ] 198 0.013% 98.157% ( 34, 51 ] 303 0.020% 98.178% ( 51, 76 ] 261 0.017% 98.195% ( 76, 110 ] 124 0.008% 98.203% ( 110, 170 ] 1448 0.097% 98.300% ( 170, 250 ] 5156 0.344% 98.645% ( 250, 380 ] 5061 0.338% 98.983% ( 380, 580 ] 6386 0.427% 99.409% ( 580, 870 ] 6076 0.406% 99.815% ( 870, 1300 ] 2899 0.194% 100.009% ( 1300, 1900 ] 1212 0.081% 100.090% ( 1900, 2900 ] 1419 0.095% 100.185% ( 2900, 4400 ] 1647 0.110% 100.295% ( 4400, 6600 ] 383 0.026% 100.320% ( 6600, 9900 ] 138 0.009% 100.329% ( 9900, 14000 ] 32 0.002% 100.331% ( 14000, 22000 ] 57 0.004% 100.335% ( 22000, 33000 ] 34 0.002% 100.338% ( 33000, 50000 ] 16 0.001% 100.339% ( 50000, 75000 ] 3 0.000% 100.339% ** Level 5 read latency histogram (micros): Count: 19564 Average: 802.1571 StdDev: 1619.79 Min: 97 Median: 475.7599 Max: 75022 Percentiles: P50: 475.76 P75: 791.57 P99: 5269.62 P99.9: 22436.00 P99.99: 47967.65 ------------------------------------------------------ ( 76, 110 ] 26 0.133% 0.133% ( 110, 170 ] 998 5.101% 5.234% # ( 170, 250 ] 3561 18.202% 23.436% #### ( 250, 380 ] 3187 16.290% 39.726% ### ( 380, 580 ] 4198 21.458% 61.184% #### ( 580, 870 ] 3705 18.938% 80.122% #### ( 870, 1300 ] 1659 8.480% 88.602% ## ( 1300, 1900 ] 580 2.965% 91.566% # ( 1900, 2900 ] 610 3.118% 94.684% # ( 2900, 4400 ] 774 3.956% 98.640% # ( 4400, 6600 ] 178 0.910% 99.550% ( 6600, 9900 ] 46 0.235% 99.785% ( 9900, 14000 ] 9 0.046% 99.831% ( 14000, 22000 ] 13 0.066% 99.898% ( 22000, 33000 ] 11 0.056% 99.954% ( 33000, 50000 ] 8 0.041% 99.995% ( 50000, 75000 ] 1 0.005% 100.000% ( 75000, 110000 ] 1 0.005% 100.005% ** DB Stats ** Uptime(secs): 12.4 total, 5.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 159.03 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.301 H:M:S, 26.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 174.65 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.332 H:M:S, 23.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/09-13:53:59 ... thread 0: (1000000,3000000) ops and (273270.0,187931.4) ops/second in (3.659384,15.963274) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/8 2.81 GB 237.7 2.2 0.0 2.2 5.0 2.8 0.0 1.8 62.2 141.4 36.38 18.07 49 0.742 2299K 12K 0.0 0.0 L1 2/0 119.74 MB 12.0 0.3 0.2 0.1 0.3 0.2 0.0 1.3 102.1 101.5 3.05 1.63 1 3.046 315K 1908 0.0 0.0 L3 17/0 984.25 MB 1.0 2.8 1.9 0.9 2.7 1.8 0.1 1.4 108.3 104.3 26.72 15.04 14 1.909 2137K 11K 0.0 0.0 L4 148/0 9.57 GB 1.0 8.6 2.7 5.9 7.5 1.6 0.9 2.8 139.6 121.7 62.97 39.24 34 1.852 2078K 86K 0.0 0.0 L5 154/0 11.63 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 338/8 25.09 GB 0.0 13.9 4.9 9.1 15.5 6.5 3.5 5.5 110.4 123.2 129.12 73.99 98 1.318 6831K 111K 0.0 0.0 Int 0/0 0.00 KB 0.0 13.9 4.9 9.1 15.5 6.5 3.5 5.5 110.4 123.2 129.12 73.99 98 1.318 6831K 111K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 13.9 4.9 9.1 12.7 3.6 0.0 0.0 124.3 113.5 114.68 65.62 57 2.012 6831K 111K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 200.3 14.44 8.37 41 0.352 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.0 total, 16.0 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 15.53 GB write, 991.86 MB/s write, 13.93 GB read, 889.24 MB/s read, 129.1 seconds Interval compaction: 15.53 GB write, 996.23 MB/s write, 13.93 GB read, 893.16 MB/s read, 129.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 2, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 2, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 2, total-stops: 27, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 717621 Average: 29.4183 StdDev: 293.46 Min: 1 Median: 1.6894 Max: 52990 Percentiles: P50: 1.69 P75: 2.47 P99: 622.82 P99.9: 2266.85 P99.99: 3468.59 ------------------------------------------------------ [ 0, 1 ] 85823 11.959% 11.959% ## ( 1, 2 ] 396003 55.183% 67.142% ########### ( 2, 3 ] 119623 16.669% 83.812% ### ( 3, 4 ] 37154 5.177% 88.989% # ( 4, 6 ] 31654 4.411% 93.400% # ( 6, 10 ] 10036 1.399% 94.798% ( 10, 15 ] 2072 0.289% 95.087% ( 15, 22 ] 268 0.037% 95.124% ( 22, 34 ] 55 0.008% 95.132% ( 34, 51 ] 30 0.004% 95.136% ( 51, 76 ] 22 0.003% 95.139% ( 76, 110 ] 199 0.028% 95.167% ( 110, 170 ] 4394 0.612% 95.779% ( 170, 250 ] 10615 1.479% 97.259% ( 250, 380 ] 6402 0.892% 98.151% ( 380, 580 ] 5449 0.759% 98.910% ( 580, 870 ] 4380 0.610% 99.520% ( 870, 1300 ] 1924 0.268% 99.788% ( 1300, 1900 ] 582 0.081% 99.870% ( 1900, 2900 ] 598 0.083% 99.953% ( 2900, 4400 ] 705 0.098% 100.051% ( 4400, 6600 ] 116 0.016% 100.067% ( 6600, 9900 ] 39 0.005% 100.073% ( 9900, 14000 ] 16 0.002% 100.075% ( 14000, 22000 ] 22 0.003% 100.078% ( 22000, 33000 ] 14 0.002% 100.080% ( 33000, 50000 ] 6 0.001% 100.081% ( 50000, 75000 ] 2 0.000% 100.081% ** Level 1 read latency histogram (micros): Count: 70186 Average: 23.7633 StdDev: 282.56 Min: 1 Median: 2.2786 Max: 43249 Percentiles: P50: 2.28 P75: 3.31 P99: 547.01 P99.9: 3577.59 P99.99: 9067.33 ------------------------------------------------------ [ 0, 1 ] 536 0.764% 0.764% ( 1, 2 ] 28654 40.826% 41.589% ######## ( 2, 3 ] 21190 30.191% 71.781% ###### ( 3, 4 ] 7345 10.465% 82.246% ## ( 4, 6 ] 7926 11.293% 93.539% ## ( 6, 10 ] 2082 2.966% 96.505% # ( 10, 15 ] 385 0.549% 97.054% ( 15, 22 ] 38 0.054% 97.108% ( 22, 34 ] 10 0.014% 97.122% ( 34, 51 ] 10 0.014% 97.136% ( 51, 76 ] 12 0.017% 97.153% ( 76, 110 ] 9 0.013% 97.166% ( 110, 170 ] 232 0.331% 97.497% ( 170, 250 ] 466 0.664% 98.161% ( 250, 380 ] 276 0.393% 98.554% ( 380, 580 ] 375 0.534% 99.088% ( 580, 870 ] 308 0.439% 99.527% ( 870, 1300 ] 141 0.201% 99.728% ( 1300, 1900 ] 50 0.071% 99.799% ( 1900, 2900 ] 41 0.058% 99.858% ( 2900, 4400 ] 66 0.094% 99.952% ( 4400, 6600 ] 21 0.030% 99.981% ( 6600, 9900 ] 8 0.011% 99.993% ( 9900, 14000 ] 1 0.001% 99.994% ( 14000, 22000 ] 3 0.004% 99.999% ( 33000, 50000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 399473 Average: 15.8843 StdDev: 314.28 Min: 1 Median: 2.2728 Max: 45056 Percentiles: P50: 2.27 P75: 3.33 P99: 13.15 P99.9: 1197.62 P99.99: 1982.33 ------------------------------------------------------ [ 0, 1 ] 3917 0.981% 0.981% ( 1, 2 ] 163997 41.053% 42.034% ######## ( 2, 3 ] 116637 29.198% 71.232% ###### ( 3, 4 ] 44950 11.252% 82.484% ## ( 4, 6 ] 51028 12.774% 95.258% ### ( 6, 10 ] 12764 3.195% 98.453% # ( 10, 15 ] 3464 0.867% 99.320% ( 15, 22 ] 598 0.150% 99.470% ( 22, 34 ] 85 0.021% 99.491% ( 34, 51 ] 87 0.022% 99.513% ( 51, 76 ] 76 0.019% 99.532% ( 76, 110 ] 21 0.005% 99.537% ( 110, 170 ] 22 0.006% 99.543% ( 170, 250 ] 56 0.014% 99.557% ( 250, 380 ] 197 0.049% 99.606% ( 380, 580 ] 433 0.108% 99.714% ( 580, 870 ] 484 0.121% 99.836% ( 870, 1300 ] 338 0.085% 99.920% ( 1300, 1900 ] 256 0.064% 99.984% ( 1900, 2900 ] 280 0.070% 100.054% ( 2900, 4400 ] 291 0.073% 100.127% ( 4400, 6600 ] 112 0.028% 100.155% ( 6600, 9900 ] 30 0.008% 100.163% ( 9900, 14000 ] 15 0.004% 100.166% ( 14000, 22000 ] 17 0.004% 100.171% ( 22000, 33000 ] 20 0.005% 100.176% ( 33000, 50000 ] 5 0.001% 100.177% ** Level 3 read latency histogram (micros): Count: 911102 Average: 18.2503 StdDev: 265.17 Min: 1 Median: 2.3321 Max: 51076 Percentiles: P50: 2.33 P75: 3.40 P99: 235.92 P99.9: 1199.20 P99.99: 1841.30 ------------------------------------------------------ [ 0, 1 ] 7651 0.840% 0.840% ( 1, 2 ] 354934 38.957% 39.796% ######## ( 2, 3 ] 279943 30.726% 70.522% ###### ( 3, 4 ] 103003 11.305% 81.827% ## ( 4, 6 ] 115012 12.623% 94.451% ### ( 6, 10 ] 28990 3.182% 97.633% # ( 10, 15 ] 6535 0.717% 98.350% ( 15, 22 ] 902 0.099% 98.449% ( 22, 34 ] 155 0.017% 98.466% ( 34, 51 ] 258 0.028% 98.494% ( 51, 76 ] 191 0.021% 98.515% ( 76, 110 ] 156 0.017% 98.532% ( 110, 170 ] 1732 0.190% 98.722% ( 170, 250 ] 3069 0.337% 99.059% ( 250, 380 ] 1807 0.198% 99.258% ( 380, 580 ] 2505 0.275% 99.533% ( 580, 870 ] 2420 0.266% 99.798% ( 870, 1300 ] 1212 0.133% 99.931% ( 1300, 1900 ] 594 0.065% 99.996% ( 1900, 2900 ] 661 0.073% 100.069% ( 2900, 4400 ] 801 0.088% 100.157% ( 4400, 6600 ] 174 0.019% 100.176% ( 6600, 9900 ] 52 0.006% 100.182% ( 9900, 14000 ] 21 0.002% 100.184% ( 14000, 22000 ] 29 0.003% 100.187% ( 22000, 33000 ] 17 0.002% 100.189% ( 33000, 50000 ] 7 0.001% 100.190% ( 50000, 75000 ] 1 0.000% 100.190% ** Level 4 read latency histogram (micros): Count: 1525570 Average: 26.5251 StdDev: 298.76 Min: 1 Median: 2.3646 Max: 53019 Percentiles: P50: 2.36 P75: 3.54 P99: 407.26 P99.9: 1055.05 P99.99: 1258.07 ------------------------------------------------------ [ 0, 1 ] 14116 0.925% 0.925% ( 1, 2 ] 584765 38.331% 39.256% ######## ( 2, 3 ] 449524 29.466% 68.722% ###### ( 3, 4 ] 175958 11.534% 80.256% ## ( 4, 6 ] 186591 12.231% 92.487% ## ( 6, 10 ] 46596 3.054% 95.541% # ( 10, 15 ] 11137 0.730% 96.271% ( 15, 22 ] 1349 0.088% 96.360% ( 22, 34 ] 198 0.013% 96.373% ( 34, 51 ] 303 0.020% 96.393% ( 51, 76 ] 261 0.017% 96.410% ( 76, 110 ] 681 0.045% 96.454% ( 110, 170 ] 10689 0.701% 97.155% ( 170, 250 ] 19325 1.267% 98.422% ( 250, 380 ] 7827 0.513% 98.935% ( 380, 580 ] 7294 0.478% 99.413% ( 580, 870 ] 6179 0.405% 99.818% ( 870, 1300 ] 2908 0.191% 100.009% ( 1300, 1900 ] 1212 0.079% 100.088% ( 1900, 2900 ] 1419 0.093% 100.181% ( 2900, 4400 ] 1647 0.108% 100.289% ( 4400, 6600 ] 383 0.025% 100.314% ( 6600, 9900 ] 138 0.009% 100.323% ( 9900, 14000 ] 32 0.002% 100.325% ( 14000, 22000 ] 57 0.004% 100.329% ( 22000, 33000 ] 34 0.002% 100.331% ( 33000, 50000 ] 16 0.001% 100.332% ( 50000, 75000 ] 3 0.000% 100.332% ** Level 5 read latency histogram (micros): Count: 39321 Average: 499.5374 StdDev: 1182.68 Min: 90 Median: 237.8768 Max: 75022 Percentiles: P50: 237.88 P75: 490.25 P99: 4153.47 P99.9: 11120.43 P99.99: 43769.29 ------------------------------------------------------ ( 76, 110 ] 421 1.071% 1.071% ( 110, 170 ] 7507 19.092% 20.162% #### ( 170, 250 ] 13828 35.167% 55.329% ####### ( 250, 380 ] 5108 12.991% 68.320% ### ( 380, 580 ] 4765 12.118% 80.438% ## ( 580, 870 ] 3800 9.664% 90.102% ## ( 870, 1300 ] 1662 4.227% 94.329% # ( 1300, 1900 ] 580 1.475% 95.804% ( 1900, 2900 ] 610 1.551% 97.355% ( 2900, 4400 ] 774 1.968% 99.324% ( 4400, 6600 ] 178 0.453% 99.776% ( 6600, 9900 ] 46 0.117% 99.893% ( 9900, 14000 ] 9 0.023% 99.916% ( 14000, 22000 ] 13 0.033% 99.949% ( 22000, 33000 ] 11 0.028% 99.977% ( 33000, 50000 ] 8 0.020% 99.997% ( 50000, 75000 ] 1 0.003% 100.000% ( 75000, 110000 ] 1 0.003% 100.003% ** DB Stats ** Uptime(secs): 16.0 total, 3.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 184.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.301 H:M:S, 20.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 268.94 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/12/09-13:54:03 ... thread 0: (1000000,4000000) ops and (255023.6,201161.9) ops/second in (3.921206,19.884480) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/17 3.73 GB 0.0 3.4 0.0 3.4 7.2 3.7 0.0 1.9 79.6 165.9 44.25 25.52 69 0.641 3576K 18K 0.0 0.0 L1 2/2 119.74 MB 0.0 0.3 0.2 0.1 0.3 0.2 0.0 1.3 102.1 101.5 3.05 1.63 1 3.046 315K 1908 0.0 0.0 L3 17/0 984.25 MB 1.0 2.8 1.9 0.9 2.7 1.8 0.1 1.4 108.3 104.3 26.72 15.04 14 1.909 2137K 11K 0.0 0.0 L4 148/0 9.57 GB 1.0 8.6 2.7 5.9 7.5 1.6 0.9 2.8 139.6 121.7 62.97 39.24 34 1.852 2078K 86K 0.0 0.0 L5 154/0 11.63 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.5 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 338/19 26.01 GB 0.0 15.2 4.9 10.3 17.7 7.4 3.5 4.7 113.3 132.1 137.00 81.44 118 1.161 8108K 117K 0.0 0.0 Int 0/0 0.00 KB 0.0 15.2 4.9 10.3 17.7 7.4 3.5 4.7 113.3 132.1 137.00 81.44 118 1.161 8108K 117K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 15.2 4.9 10.3 13.9 3.6 0.0 0.0 129.7 119.2 119.69 70.49 62 1.930 8108K 117K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 3.7 3.7 0.0 0.0 0.0 221.7 17.31 10.95 56 0.309 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 20.0 total, 19.9 interval Flush(GB): cumulative 3.747, interval 3.747 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 17.68 GB write, 907.00 MB/s write, 15.15 GB read, 777.57 MB/s read, 137.0 seconds Interval compaction: 17.68 GB write, 910.22 MB/s write, 15.15 GB read, 780.33 MB/s read, 137.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 27, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 4, total-stops: 27, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 997915 Average: 24.0208 StdDev: 249.95 Min: 1 Median: 1.6214 Max: 52990 Percentiles: P50: 1.62 P75: 2.26 P99: 501.64 P99.9: 1695.64 P99.99: 3115.34 ------------------------------------------------------ [ 0, 1 ] 151338 15.165% 15.165% ### ( 1, 2 ] 559426 56.059% 71.225% ########### ( 2, 3 ] 143863 14.416% 85.641% ### ( 3, 4 ] 44646 4.474% 90.115% # ( 4, 6 ] 37431 3.751% 93.866% # ( 6, 10 ] 11302 1.133% 94.999% ( 10, 15 ] 2454 0.246% 95.245% ( 15, 22 ] 338 0.034% 95.278% ( 22, 34 ] 87 0.009% 95.287% ( 34, 51 ] 34 0.003% 95.291% ( 51, 76 ] 25 0.003% 95.293% ( 76, 110 ] 513 0.051% 95.344% ( 110, 170 ] 9049 0.907% 96.251% ( 170, 250 ] 16791 1.683% 97.934% ( 250, 380 ] 7160 0.717% 98.651% ( 380, 580 ] 5720 0.573% 99.225% ( 580, 870 ] 4426 0.444% 99.668% ( 870, 1300 ] 1929 0.193% 99.861% ( 1300, 1900 ] 584 0.059% 99.920% ( 1900, 2900 ] 598 0.060% 99.980% ( 2900, 4400 ] 705 0.071% 100.051% ( 4400, 6600 ] 116 0.012% 100.062% ( 6600, 9900 ] 39 0.004% 100.066% ( 9900, 14000 ] 16 0.002% 100.068% ( 14000, 22000 ] 22 0.002% 100.070% ( 22000, 33000 ] 14 0.001% 100.071% ( 33000, 50000 ] 6 0.001% 100.072% ( 50000, 75000 ] 2 0.000% 100.072% ** Level 1 read latency histogram (micros): Count: 70777 Average: 25.1505 StdDev: 281.84 Min: 1 Median: 2.2925 Max: 43249 Percentiles: P50: 2.29 P75: 3.37 P99: 545.58 P99.9: 3564.16 P99.99: 9042.95 ------------------------------------------------------ [ 0, 1 ] 536 0.757% 0.757% ( 1, 2 ] 28654 40.485% 41.242% ######## ( 2, 3 ] 21190 29.939% 71.181% ###### ( 3, 4 ] 7345 10.378% 81.559% ## ( 4, 6 ] 7926 11.199% 92.758% ## ( 6, 10 ] 2082 2.942% 95.699% # ( 10, 15 ] 385 0.544% 96.243% ( 15, 22 ] 38 0.054% 96.297% ( 22, 34 ] 10 0.014% 96.311% ( 34, 51 ] 10 0.014% 96.325% ( 51, 76 ] 12 0.017% 96.342% ( 76, 110 ] 24 0.034% 96.376% ( 110, 170 ] 440 0.622% 96.998% ( 170, 250 ] 782 1.105% 98.102% ( 250, 380 ] 314 0.444% 98.546% ( 380, 580 ] 388 0.548% 99.094% ( 580, 870 ] 309 0.437% 99.531% ( 870, 1300 ] 141 0.199% 99.730% ( 1300, 1900 ] 50 0.071% 99.801% ( 1900, 2900 ] 41 0.058% 99.859% ( 2900, 4400 ] 66 0.093% 99.952% ( 4400, 6600 ] 21 0.030% 99.982% ( 6600, 9900 ] 8 0.011% 99.993% ( 9900, 14000 ] 1 0.001% 99.994% ( 14000, 22000 ] 3 0.004% 99.999% ( 33000, 50000 ] 1 0.001% 100.000% ** Level 2 read latency histogram (micros): Count: 399473 Average: 15.8843 StdDev: 314.28 Min: 1 Median: 2.2728 Max: 45056 Percentiles: P50: 2.27 P75: 3.33 P99: 13.15 P99.9: 1197.62 P99.99: 1982.33 ------------------------------------------------------ [ 0, 1 ] 3917 0.981% 0.981% ( 1, 2 ] 163997 41.053% 42.034% ######## ( 2, 3 ] 116637 29.198% 71.232% ###### ( 3, 4 ] 44950 11.252% 82.484% ## ( 4, 6 ] 51028 12.774% 95.258% ### ( 6, 10 ] 12764 3.195% 98.453% # ( 10, 15 ] 3464 0.867% 99.320% ( 15, 22 ] 598 0.150% 99.470% ( 22, 34 ] 85 0.021% 99.491% ( 34, 51 ] 87 0.022% 99.513% ( 51, 76 ] 76 0.019% 99.532% ( 76, 110 ] 21 0.005% 99.537% ( 110, 170 ] 22 0.006% 99.543% ( 170, 250 ] 56 0.014% 99.557% ( 250, 380 ] 197 0.049% 99.606% ( 380, 580 ] 433 0.108% 99.714% ( 580, 870 ] 484 0.121% 99.836% ( 870, 1300 ] 338 0.085% 99.920% ( 1300, 1900 ] 256 0.064% 99.984% ( 1900, 2900 ] 280 0.070% 100.054% ( 2900, 4400 ] 291 0.073% 100.127% ( 4400, 6600 ] 112 0.028% 100.155% ( 6600, 9900 ] 30 0.008% 100.163% ( 9900, 14000 ] 15 0.004% 100.166% ( 14000, 22000 ] 17 0.004% 100.171% ( 22000, 33000 ] 20 0.005% 100.176% ( 33000, 50000 ] 5 0.001% 100.177% ** Level 3 read latency histogram (micros): Count: 915510 Average: 19.0678 StdDev: 264.82 Min: 1 Median: 2.3400 Max: 51076 Percentiles: P50: 2.34 P75: 3.43 P99: 246.53 P99.9: 1197.64 P99.99: 1840.86 ------------------------------------------------------ [ 0, 1 ] 7651 0.836% 0.836% ( 1, 2 ] 354934 38.769% 39.605% ######## ( 2, 3 ] 279943 30.578% 70.183% ###### ( 3, 4 ] 103003 11.251% 81.433% ## ( 4, 6 ] 115012 12.563% 93.996% ### ( 6, 10 ] 28990 3.167% 97.163% # ( 10, 15 ] 6535 0.714% 97.876% ( 15, 22 ] 902 0.099% 97.975% ( 22, 34 ] 155 0.017% 97.992% ( 34, 51 ] 258 0.028% 98.020% ( 51, 76 ] 191 0.021% 98.041% ( 76, 110 ] 248 0.027% 98.068% ( 110, 170 ] 3412 0.373% 98.441% ( 170, 250 ] 5353 0.585% 99.025% ( 250, 380 ] 2073 0.226% 99.252% ( 380, 580 ] 2577 0.281% 99.533% ( 580, 870 ] 2434 0.266% 99.799% ( 870, 1300 ] 1212 0.132% 99.932% ( 1300, 1900 ] 594 0.065% 99.996% ( 1900, 2900 ] 661 0.072% 100.069% ( 2900, 4400 ] 801 0.087% 100.156% ( 4400, 6600 ] 174 0.019% 100.175% ( 6600, 9900 ] 52 0.006% 100.181% ( 9900, 14000 ] 21 0.002% 100.183% ( 14000, 22000 ] 29 0.003% 100.186% ( 22000, 33000 ] 17 0.002% 100.188% ( 33000, 50000 ] 7 0.001% 100.189% ( 50000, 75000 ] 1 0.000% 100.189% ** Level 4 read latency histogram (micros): Count: 1555411 Average: 29.6248 StdDev: 296.85 Min: 1 Median: 2.3978 Max: 53019 Percentiles: P50: 2.40 P75: 3.67 P99: 414.99 P99.9: 1051.23 P99.99: 1257.73 ------------------------------------------------------ [ 0, 1 ] 14116 0.908% 0.908% ( 1, 2 ] 584775 37.596% 38.504% ######## ( 2, 3 ] 449553 28.903% 67.406% ###### ( 3, 4 ] 175994 11.315% 78.721% ## ( 4, 6 ] 186644 12.000% 90.721% ## ( 6, 10 ] 46603 2.996% 93.717% # ( 10, 15 ] 11142 0.716% 94.433% ( 15, 22 ] 1351 0.087% 94.520% ( 22, 34 ] 198 0.013% 94.533% ( 34, 51 ] 303 0.019% 94.552% ( 51, 76 ] 261 0.017% 94.569% ( 76, 110 ] 1425 0.092% 94.661% ( 110, 170 ] 22221 1.429% 96.089% ( 170, 250 ] 34109 2.193% 98.282% ( 250, 380 ] 9784 0.629% 98.911% ( 380, 580 ] 7876 0.506% 99.418% ( 580, 870 ] 6272 0.403% 99.821% ( 870, 1300 ] 2915 0.187% 100.008% ( 1300, 1900 ] 1212 0.078% 100.086% ( 1900, 2900 ] 1420 0.091% 100.178% ( 2900, 4400 ] 1647 0.106% 100.284% ( 4400, 6600 ] 383 0.025% 100.308% ( 6600, 9900 ] 138 0.009% 100.317% ( 9900, 14000 ] 32 0.002% 100.319% ( 14000, 22000 ] 57 0.004% 100.323% ( 22000, 33000 ] 34 0.002% 100.325% ( 33000, 50000 ] 16 0.001% 100.326% ( 50000, 75000 ] 3 0.000% 100.326% ** Level 5 read latency histogram (micros): Count: 59549 Average: 394.1837 StdDev: 972.91 Min: 90 Median: 215.3942 Max: 75022 Percentiles: P50: 215.39 P75: 340.46 P99: 3761.45 P99.9: 8641.05 P99.99: 39470.84 ------------------------------------------------------ ( 76, 110 ] 919 1.543% 1.543% ( 110, 170 ] 15210 25.542% 27.085% ##### ( 170, 250 ] 24048 40.384% 67.469% ######## ( 250, 380 ] 6445 10.823% 78.292% ## ( 380, 580 ] 5180 8.699% 86.991% ## ( 580, 870 ] 3849 6.464% 93.454% # ( 870, 1300 ] 1668 2.801% 96.255% # ( 1300, 1900 ] 580 0.974% 97.229% ( 1900, 2900 ] 610 1.024% 98.254% ( 2900, 4400 ] 774 1.300% 99.553% ( 4400, 6600 ] 178 0.299% 99.852% ( 6600, 9900 ] 46 0.077% 99.929% ( 9900, 14000 ] 9 0.015% 99.945% ( 14000, 22000 ] 13 0.022% 99.966% ( 22000, 33000 ] 11 0.018% 99.985% ( 33000, 50000 ] 8 0.013% 99.998% ( 50000, 75000 ] 1 0.002% 100.000% ( 75000, 110000 ] 1 0.002% 100.002% ** DB Stats ** Uptime(secs): 20.0 total, 3.9 interval Cumulative writes: 0 writes, 4000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 3.84 GB, 197.25 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.553 H:M:S, 17.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 251.00 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.252 H:M:S, 6.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 442.240 micros/op 9025 ops/sec 20.164 seconds 181996 operations; 8.7 MB/s (45999 of 45999 found) Microseconds per read: Count: 181996 Average: 442.2688 StdDev: 1316.40 Min: 2 Median: 241.0578 Max: 103070 Percentiles: P50: 241.06 P75: 397.11 P99: 3845.62 P99.9: 18912.56 P99.99: 49417.34 ------------------------------------------------------ ( 1, 2 ] 5 0.003% 0.003% ( 2, 3 ] 22 0.012% 0.015% ( 3, 4 ] 49 0.027% 0.042% ( 4, 6 ] 173 0.095% 0.137% ( 6, 10 ] 324 0.178% 0.315% ( 10, 15 ] 922 0.507% 0.821% ( 15, 22 ] 2492 1.369% 2.191% ( 22, 34 ] 3912 2.149% 4.340% ( 34, 51 ] 2385 1.310% 5.651% ( 51, 76 ] 441 0.242% 5.893% ( 76, 110 ] 134 0.074% 5.967% ( 110, 170 ] 14033 7.711% 13.677% ## ( 170, 250 ] 74425 40.894% 54.571% ######## ( 250, 380 ] 35314 19.404% 73.975% #### ( 380, 580 ] 21815 11.987% 85.961% ## ( 580, 870 ] 12955 7.118% 93.080% # ( 870, 1300 ] 5869 3.225% 96.304% # ( 1300, 1900 ] 2083 1.145% 97.449% ( 1900, 2900 ] 1566 0.860% 98.309% ( 2900, 4400 ] 1994 1.096% 99.405% ( 4400, 6600 ] 512 0.281% 99.686% ( 6600, 9900 ] 215 0.118% 99.804% ( 9900, 14000 ] 104 0.057% 99.862% ( 14000, 22000 ] 114 0.063% 99.924% ( 22000, 33000 ] 86 0.047% 99.971% ( 33000, 50000 ] 35 0.019% 99.991% ( 50000, 75000 ] 10 0.005% 99.996% ( 75000, 110000 ] 7 0.004% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 2760600 rocksdb.bloom.filter.full.positive COUNT : 208123 rocksdb.bloom.filter.full.true.positive COUNT : 181103 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 893 rocksdb.memtable.miss COUNT : 181103 rocksdb.l0.hit COUNT : 20451 rocksdb.l1.hit COUNT : 928 rocksdb.l2andup.hit COUNT : 159724 rocksdb.compaction.key.drop.new COUNT : 1273479 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 4076619 rocksdb.number.keys.read COUNT : 181996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 4207070808 rocksdb.bytes.read COUNT : 181996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 555 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 3553114 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 16279384327 rocksdb.compact.write.bytes COUNT : 14967236223 rocksdb.flush.write.bytes COUNT : 4088989721 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 28 rocksdb.number.superversion_acquires COUNT : 819 rocksdb.number.superversion_releases COUNT : 798 rocksdb.number.superversion_cleanups COUNT : 202 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 216 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 4062528512 rocksdb.memtable.garbage.bytes.at.flush COUNT : 7553024 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 16762230970 rocksdb.non.last.level.read.count COUNT : 4022335 rocksdb.block.checksum.compute.count COUNT : 4155030 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 240.338883 P95 : 1124.987355 P99 : 3845.338685 P100 : 103068.000000 COUNT : 181996 SUM : 80315488 rocksdb.db.write.micros P50 : 1.572273 P95 : 5.053846 P99 : 6.925582 P100 : 671586.000000 COUNT : 4076619 SUM : 14833395 rocksdb.compaction.times.micros P50 : 1740909.090909 P95 : 3992000.000000 P99 : 5818000.000000 P100 : 6257109.000000 COUNT : 62 SUM : 119687342 rocksdb.compaction.times.cpu_micros P50 : 1159200.000000 P95 : 1845555.555556 P99 : 2024419.000000 P100 : 2024419.000000 COUNT : 62 SUM : 70485238 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 919.615385 P95 : 8373.750000 P99 : 11663.000000 P100 : 11812.000000 COUNT : 57 SUM : 105949 rocksdb.compaction.outfile.sync.micros P50 : 2842.857143 P95 : 7700.000000 P99 : 9900.000000 P100 : 24029.000000 COUNT : 200 SUM : 649980 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 231.176471 P95 : 2566.666667 P99 : 4280.000000 P100 : 13433.000000 COUNT : 140 SUM : 82591 rocksdb.table.open.io.micros P50 : 5633.333333 P95 : 18875.000000 P99 : 28325.000000 P100 : 91416.000000 COUNT : 555 SUM : 3783268 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.519675 P95 : 9.641152 P99 : 68.026335 P100 : 53024.000000 COUNT : 3943694 SUM : 62519959 rocksdb.read.block.get.micros P50 : 207.419355 P95 : 934.525852 P99 : 3462.004619 P100 : 75093.000000 COUNT : 210784 SUM : 72089008 rocksdb.write.raw.block.micros P50 : 0.538009 P95 : 1.440208 P99 : 4.413670 P100 : 84242.000000 COUNT : 4630703 SUM : 17788078 rocksdb.numfiles.in.singlecompaction P50 : 3.085366 P95 : 3.996341 P99 : 16.190000 P100 : 19.000000 COUNT : 83 SUM : 291 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.562609 P95 : 1126.436364 P99 : 2047.755102 P100 : 671409.000000 COUNT : 5176 SUM : 3553621 rocksdb.sst.read.micros P50 : 2.139262 P95 : 135.814302 P99 : 567.464724 P100 : 75022.000000 COUNT : 4022386 SUM : 120640171 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 181996 SUM : 181996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 4076619 SUM : 4207070808 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 231621.621622 P95 : 810700.000000 P99 : 964338.000000 P100 : 964338.000000 COUNT : 57 SUM : 17488527 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 216 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0