[2024-07-15 23:02:30.436783] Starting SPDK v24.09-pre git sha1 1bd83e221 / DPDK 24.03.0 initialization... [2024-07-15 23:02:30.436942] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1436818 ] Set seed to 1721077350433256 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 23:02:30.569430] app.c: 909:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 23:02:30.795221] app.c: 603:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 23:02:30.795276] app.c: 604:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1436818' to capture a snapshot of events at runtime. [2024-07-15 23:02:30.795289] app.c: 609:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-07-15 23:02:30.795303] app.c: 610:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-07-15 23:02:30.795313] app.c: 611:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1436818 for offline analysis/debug. [2024-07-15 23:02:30.795410] reactor.c: 941:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 23:02:34 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/15-23:02:41 ... thread 0: (1000000,1000000) ops and (143111.7,143111.7) ops/second in (6.987549,6.987549) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 1.32 GB 50.1 0.3 0.0 0.3 1.2 0.9 0.0 1.4 35.8 135.7 8.79 4.58 11 0.799 319K 1779 0.0 0.0 L1 7/7 389.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 23/4 1.18 GB 1.0 2.2 1.5 0.7 2.2 1.5 0.0 1.4 106.0 104.2 21.40 12.48 11 1.945 1558K 8260 0.0 0.0 L4 153/16 10.76 GB 1.0 3.3 1.8 1.5 3.1 1.6 0.2 1.7 118.1 111.5 28.39 17.33 15 1.893 839K 173 0.0 0.0 L5 136/0 10.52 GB 0.1 0.0 0.0 0.0 0.0 0.0 1.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 331/36 24.16 GB 0.0 5.8 3.3 2.5 6.4 3.9 1.5 7.5 101.3 112.5 58.57 34.38 37 1.583 2717K 10K 0.0 0.0 Int 0/0 0.00 KB 0.0 5.8 3.3 2.5 6.4 3.9 1.5 7.5 101.3 112.5 58.57 34.38 37 1.583 2717K 10K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.8 3.3 2.5 5.6 3.1 0.0 0.0 113.7 109.4 52.17 31.58 27 1.932 2717K 10K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 137.4 6.40 2.81 10 0.640 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.1 total, 7.0 interval Flush(GB): cumulative 0.859, interval 0.859 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 6.43 GB write, 931.32 MB/s write, 5.79 GB read, 839.03 MB/s read, 58.6 seconds Interval compaction: 6.43 GB write, 941.83 MB/s write, 5.79 GB read, 848.50 MB/s read, 58.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 29, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 124603 Average: 87.6278 StdDev: 663.73 Min: 1 Median: 2.1769 Max: 71980 Percentiles: P50: 2.18 P75: 3.49 P99: 1682.07 P99.9: 6260.52 P99.99: 18011.76 ------------------------------------------------------ [ 0, 1 ] 2969 2.383% 2.383% ( 1, 2 ] 53779 43.160% 45.543% ######### ( 2, 3 ] 31395 25.196% 70.739% ##### ( 3, 4 ] 10760 8.635% 79.374% ## ( 4, 6 ] 11136 8.937% 88.312% ## ( 6, 10 ] 3011 2.416% 90.728% ( 10, 15 ] 654 0.525% 91.253% ( 15, 22 ] 111 0.089% 91.342% ( 22, 34 ] 15 0.012% 91.354% ( 34, 51 ] 9 0.007% 91.361% ( 51, 76 ] 11 0.009% 91.370% ( 76, 110 ] 19 0.015% 91.385% ( 110, 170 ] 135 0.108% 91.494% ( 170, 250 ] 825 0.662% 92.156% ( 250, 380 ] 1953 1.567% 93.723% ( 380, 580 ] 2833 2.274% 95.997% ( 580, 870 ] 2497 2.004% 98.001% ( 870, 1300 ] 1010 0.811% 98.811% ( 1300, 1900 ] 369 0.296% 99.108% ( 1900, 2900 ] 380 0.305% 99.413% ( 2900, 4400 ] 489 0.392% 99.805% ( 4400, 6600 ] 140 0.112% 99.917% ( 6600, 9900 ] 60 0.048% 99.965% ( 9900, 14000 ] 17 0.014% 99.979% ( 14000, 22000 ] 27 0.022% 100.001% ( 22000, 33000 ] 7 0.006% 100.006% ( 33000, 50000 ] 4 0.003% 100.010% ( 50000, 75000 ] 3 0.002% 100.012% ** Level 1 read latency histogram (micros): Count: 532 Average: 1005.8759 StdDev: 1306.18 Min: 123 Median: 577.2028 Max: 8552 Percentiles: P50: 577.20 P75: 893.89 P99: 8144.40 P99.9: 8552.00 P99.99: 8552.00 ------------------------------------------------------ ( 110, 170 ] 5 0.940% 0.940% ( 170, 250 ] 34 6.391% 7.331% # ( 250, 380 ] 86 16.165% 23.496% ### ( 380, 580 ] 143 26.880% 50.376% ##### ( 580, 870 ] 128 24.060% 74.436% ##### ( 870, 1300 ] 54 10.150% 84.586% ## ( 1300, 1900 ] 19 3.571% 88.158% # ( 1900, 2900 ] 16 3.008% 91.165% # ( 2900, 4400 ] 31 5.827% 96.992% # ( 4400, 6600 ] 6 1.128% 98.120% ( 6600, 9900 ] 10 1.880% 100.000% ** Level 2 read latency histogram (micros): Count: 294461 Average: 16.4339 StdDev: 288.85 Min: 1 Median: 2.1773 Max: 38942 Percentiles: P50: 2.18 P75: 3.09 P99: 13.53 P99.9: 2638.30 P99.99: 4264.27 ------------------------------------------------------ [ 0, 1 ] 3434 1.166% 1.166% ( 1, 2 ] 128535 43.651% 44.817% ######### ( 2, 3 ] 86078 29.232% 74.050% ###### ( 3, 4 ] 30530 10.368% 84.418% ## ( 4, 6 ] 32726 11.114% 95.531% ## ( 6, 10 ] 8537 2.899% 98.431% # ( 10, 15 ] 2377 0.807% 99.238% ( 15, 22 ] 308 0.105% 99.343% ( 22, 34 ] 34 0.012% 99.354% ( 34, 51 ] 65 0.022% 99.376% ( 51, 76 ] 68 0.023% 99.399% ( 76, 110 ] 17 0.006% 99.405% ( 110, 170 ] 20 0.007% 99.412% ( 170, 250 ] 52 0.018% 99.429% ( 250, 380 ] 187 0.064% 99.493% ( 380, 580 ] 376 0.128% 99.621% ( 580, 870 ] 349 0.119% 99.739% ( 870, 1300 ] 197 0.067% 99.806% ( 1300, 1900 ] 137 0.047% 99.853% ( 1900, 2900 ] 189 0.064% 99.917% ( 2900, 4400 ] 237 0.080% 99.997% ( 4400, 6600 ] 92 0.031% 100.029% ( 6600, 9900 ] 72 0.024% 100.053% ( 9900, 14000 ] 16 0.005% 100.058% ( 14000, 22000 ] 12 0.004% 100.062% ( 22000, 33000 ] 8 0.003% 100.065% ( 33000, 50000 ] 2 0.001% 100.066% ** Level 3 read latency histogram (micros): Count: 536315 Average: 16.1522 StdDev: 310.21 Min: 1 Median: 2.1909 Max: 39019 Percentiles: P50: 2.19 P75: 3.06 P99: 13.34 P99.9: 1077.23 P99.99: 1916.30 ------------------------------------------------------ [ 0, 1 ] 5205 0.971% 0.971% ( 1, 2 ] 232065 43.270% 44.241% ######### ( 2, 3 ] 161761 30.162% 74.402% ###### ( 3, 4 ] 56167 10.473% 84.875% ## ( 4, 6 ] 58729 10.950% 95.826% ## ( 6, 10 ] 14919 2.782% 98.607% # ( 10, 15 ] 3152 0.588% 99.195% ( 15, 22 ] 357 0.067% 99.262% ( 22, 34 ] 59 0.011% 99.273% ( 34, 51 ] 161 0.030% 99.303% ( 51, 76 ] 107 0.020% 99.323% ( 76, 110 ] 47 0.009% 99.331% ( 110, 170 ] 70 0.013% 99.344% ( 170, 250 ] 192 0.036% 99.380% ( 250, 380 ] 579 0.108% 99.488% ( 380, 580 ] 1013 0.189% 99.677% ( 580, 870 ] 974 0.182% 99.859% ( 870, 1300 ] 460 0.086% 99.944% ( 1300, 1900 ] 240 0.045% 99.989% ( 1900, 2900 ] 268 0.050% 100.039% ( 2900, 4400 ] 332 0.062% 100.101% ( 4400, 6600 ] 99 0.018% 100.120% ( 6600, 9900 ] 97 0.018% 100.138% ( 9900, 14000 ] 27 0.005% 100.143% ( 14000, 22000 ] 41 0.008% 100.150% ( 22000, 33000 ] 20 0.004% 100.154% ( 33000, 50000 ] 5 0.001% 100.155% ** Level 4 read latency histogram (micros): Count: 595225 Average: 29.3775 StdDev: 369.45 Min: 1 Median: 2.1833 Max: 63985 Percentiles: P50: 2.18 P75: 3.14 P99: 574.81 P99.9: 2025.43 P99.99: 2857.26 ------------------------------------------------------ [ 0, 1 ] 6061 1.018% 1.018% ( 1, 2 ] 260108 43.699% 44.717% ######### ( 2, 3 ] 171518 28.816% 73.533% ###### ( 3, 4 ] 61581 10.346% 83.879% ## ( 4, 6 ] 62590 10.515% 94.394% ## ( 6, 10 ] 16399 2.755% 97.149% # ( 10, 15 ] 3373 0.567% 97.716% ( 15, 22 ] 411 0.069% 97.785% ( 22, 34 ] 71 0.012% 97.797% ( 34, 51 ] 134 0.023% 97.819% ( 51, 76 ] 105 0.018% 97.837% ( 76, 110 ] 48 0.008% 97.845% ( 110, 170 ] 187 0.031% 97.877% ( 170, 250 ] 948 0.159% 98.036% ( 250, 380 ] 2311 0.388% 98.424% ( 380, 580 ] 3519 0.591% 99.015% ( 580, 870 ] 3185 0.535% 99.550% ( 870, 1300 ] 1400 0.235% 99.786% ( 1300, 1900 ] 600 0.101% 99.886% ( 1900, 2900 ] 644 0.108% 99.995% ( 2900, 4400 ] 783 0.132% 100.126% ( 4400, 6600 ] 257 0.043% 100.169% ( 6600, 9900 ] 109 0.018% 100.188% ( 9900, 14000 ] 21 0.004% 100.191% ( 14000, 22000 ] 49 0.008% 100.199% ( 22000, 33000 ] 15 0.003% 100.202% ( 33000, 50000 ] 12 0.002% 100.204% ( 50000, 75000 ] 1 0.000% 100.204% ** Level 5 read latency histogram (micros): Count: 8994 Average: 998.8225 StdDev: 1786.53 Min: 93 Median: 575.2300 Max: 43963 Percentiles: P50: 575.23 P75: 926.21 P99: 6377.15 P99.9: 27511.00 P99.99: 43963.00 ------------------------------------------------------ ( 76, 110 ] 9 0.100% 0.100% ( 110, 170 ] 130 1.445% 1.545% ( 170, 250 ] 548 6.093% 7.638% # ( 250, 380 ] 1518 16.878% 24.516% ### ( 380, 580 ] 2348 26.106% 50.623% ##### ( 580, 870 ] 2064 22.949% 73.571% ##### ( 870, 1300 ] 983 10.930% 84.501% ## ( 1300, 1900 ] 372 4.136% 88.637% # ( 1900, 2900 ] 369 4.103% 92.740% # ( 2900, 4400 ] 466 5.181% 97.921% # ( 4400, 6600 ] 108 1.201% 99.122% ( 6600, 9900 ] 41 0.456% 99.577% ( 9900, 14000 ] 16 0.178% 99.755% ( 14000, 22000 ] 10 0.111% 99.867% ( 22000, 33000 ] 6 0.067% 99.933% ( 33000, 50000 ] 6 0.067% 100.000% ** DB Stats ** Uptime(secs): 7.1 total, 7.0 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 139.15 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.070 H:M:S, 29.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 140.71 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.070 H:M:S, 29.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 6 num-running-flushes: 1 2024/07/15-23:02:45 ... thread 0: (1000000,2000000) ops and (265029.0,185861.2) ops/second in (3.773172,10.760721) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/9 2.30 GB 162.6 1.5 0.0 1.5 3.3 1.8 0.0 1.8 83.1 186.5 18.16 12.11 31 0.586 1532K 8879 0.0 0.0 L1 7/7 389.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 957.67 MB 1.0 2.2 1.5 0.7 2.2 1.5 0.0 1.4 106.0 104.2 21.40 12.48 11 1.945 1558K 8260 0.0 0.0 L4 141/0 9.66 GB 1.0 4.7 2.0 2.7 4.3 1.6 0.2 2.1 132.7 120.8 36.62 23.44 19 1.928 1091K 173 0.0 0.0 L5 152/0 11.63 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 335/16 24.89 GB 0.0 8.4 3.5 4.9 9.8 4.9 2.6 5.3 113.4 131.8 76.18 48.02 61 1.249 4182K 17K 0.0 0.0 Int 0/0 0.00 KB 0.0 8.4 3.5 4.9 9.8 4.9 2.6 5.3 113.4 131.8 76.18 48.02 61 1.249 4182K 17K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.4 3.5 4.9 8.0 3.1 0.0 0.0 129.9 122.6 66.48 42.39 35 1.900 4182K 17K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 194.6 9.69 5.64 26 0.373 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 10.8 total, 10.8 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.81 GB write, 925.80 MB/s write, 8.43 GB read, 796.32 MB/s read, 76.2 seconds Interval compaction: 9.81 GB write, 932.59 MB/s write, 8.43 GB read, 802.16 MB/s read, 76.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 29, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 495157 Average: 34.3028 StdDev: 339.86 Min: 0 Median: 1.6844 Max: 71980 Percentiles: P50: 1.68 P75: 2.49 P99: 597.65 P99.9: 2704.55 P99.99: 3984.27 ------------------------------------------------------ [ 0, 1 ] 59903 12.098% 12.098% ## ( 1, 2 ] 274225 55.381% 67.479% ########### ( 2, 3 ] 76583 15.466% 82.946% ### ( 3, 4 ] 23976 4.842% 87.788% # ( 4, 6 ] 19902 4.019% 91.807% # ( 6, 10 ] 6609 1.335% 93.142% ( 10, 15 ] 1003 0.203% 93.344% ( 15, 22 ] 185 0.037% 93.382% ( 22, 34 ] 30 0.006% 93.388% ( 34, 51 ] 36 0.007% 93.395% ( 51, 76 ] 13 0.003% 93.398% ( 76, 110 ] 234 0.047% 93.445% ( 110, 170 ] 5138 1.038% 94.483% ( 170, 250 ] 12373 2.499% 96.981% ( 250, 380 ] 5527 1.116% 98.098% ( 380, 580 ] 4296 0.868% 98.965% ( 580, 870 ] 2850 0.576% 99.541% ( 870, 1300 ] 1063 0.215% 99.755% ( 1300, 1900 ] 391 0.079% 99.834% ( 1900, 2900 ] 405 0.082% 99.916% ( 2900, 4400 ] 507 0.102% 100.019% ( 4400, 6600 ] 142 0.029% 100.047% ( 6600, 9900 ] 60 0.012% 100.059% ( 9900, 14000 ] 17 0.003% 100.063% ( 14000, 22000 ] 27 0.005% 100.068% ( 22000, 33000 ] 7 0.001% 100.070% ( 33000, 50000 ] 4 0.001% 100.070% ( 50000, 75000 ] 3 0.001% 100.071% ** Level 1 read latency histogram (micros): Count: 1531 Average: 499.1973 StdDev: 859.81 Min: 100 Median: 243.3566 Max: 8552 Percentiles: P50: 243.36 P75: 475.85 P99: 4653.00 P99.9: 8552.00 P99.99: 8552.00 ------------------------------------------------------ ( 76, 110 ] 11 0.718% 0.718% ( 110, 170 ] 230 15.023% 15.741% ### ( 170, 250 ] 572 37.361% 53.103% ####### ( 250, 380 ] 237 15.480% 68.583% ### ( 380, 580 ] 205 13.390% 81.973% ### ( 580, 870 ] 137 8.948% 90.921% ## ( 870, 1300 ] 56 3.658% 94.579% # ( 1300, 1900 ] 19 1.241% 95.820% ( 1900, 2900 ] 17 1.110% 96.930% ( 2900, 4400 ] 31 2.025% 98.955% ( 4400, 6600 ] 6 0.392% 99.347% ( 6600, 9900 ] 10 0.653% 100.000% ** Level 2 read latency histogram (micros): Count: 294461 Average: 16.4339 StdDev: 288.85 Min: 1 Median: 2.1773 Max: 38942 Percentiles: P50: 2.18 P75: 3.09 P99: 13.53 P99.9: 2638.30 P99.99: 4264.27 ------------------------------------------------------ [ 0, 1 ] 3434 1.166% 1.166% ( 1, 2 ] 128535 43.651% 44.817% ######### ( 2, 3 ] 86078 29.232% 74.050% ###### ( 3, 4 ] 30530 10.368% 84.418% ## ( 4, 6 ] 32726 11.114% 95.531% ## ( 6, 10 ] 8537 2.899% 98.431% # ( 10, 15 ] 2377 0.807% 99.238% ( 15, 22 ] 308 0.105% 99.343% ( 22, 34 ] 34 0.012% 99.354% ( 34, 51 ] 65 0.022% 99.376% ( 51, 76 ] 68 0.023% 99.399% ( 76, 110 ] 17 0.006% 99.405% ( 110, 170 ] 20 0.007% 99.412% ( 170, 250 ] 52 0.018% 99.429% ( 250, 380 ] 187 0.064% 99.493% ( 380, 580 ] 376 0.128% 99.621% ( 580, 870 ] 349 0.119% 99.739% ( 870, 1300 ] 197 0.067% 99.806% ( 1300, 1900 ] 137 0.047% 99.853% ( 1900, 2900 ] 189 0.064% 99.917% ( 2900, 4400 ] 237 0.080% 99.997% ( 4400, 6600 ] 92 0.031% 100.029% ( 6600, 9900 ] 72 0.024% 100.053% ( 9900, 14000 ] 16 0.005% 100.058% ( 14000, 22000 ] 12 0.004% 100.062% ( 22000, 33000 ] 8 0.003% 100.065% ( 33000, 50000 ] 2 0.001% 100.066% ** Level 3 read latency histogram (micros): Count: 559637 Average: 16.9446 StdDev: 304.37 Min: 1 Median: 2.1913 Max: 39019 Percentiles: P50: 2.19 P75: 3.08 P99: 184.66 P99.9: 1060.73 P99.99: 1895.15 ------------------------------------------------------ [ 0, 1 ] 5580 0.997% 0.997% ( 1, 2 ] 242242 43.286% 44.283% ######### ( 2, 3 ] 167265 29.888% 74.171% ###### ( 3, 4 ] 58072 10.377% 84.547% ## ( 4, 6 ] 60404 10.793% 95.341% ## ( 6, 10 ] 15406 2.753% 98.094% # ( 10, 15 ] 3185 0.569% 98.663% ( 15, 22 ] 362 0.065% 98.728% ( 22, 34 ] 62 0.011% 98.739% ( 34, 51 ] 170 0.030% 98.769% ( 51, 76 ] 107 0.019% 98.788% ( 76, 110 ] 73 0.013% 98.801% ( 110, 170 ] 775 0.138% 98.940% ( 170, 250 ] 1843 0.329% 99.269% ( 250, 380 ] 1040 0.186% 99.455% ( 380, 580 ] 1228 0.219% 99.674% ( 580, 870 ] 1054 0.188% 99.863% ( 870, 1300 ] 472 0.084% 99.947% ( 1300, 1900 ] 243 0.043% 99.990% ( 1900, 2900 ] 272 0.049% 100.039% ( 2900, 4400 ] 332 0.059% 100.098% ( 4400, 6600 ] 99 0.018% 100.116% ( 6600, 9900 ] 97 0.017% 100.133% ( 9900, 14000 ] 27 0.005% 100.138% ( 14000, 22000 ] 41 0.007% 100.145% ( 22000, 33000 ] 20 0.004% 100.149% ( 33000, 50000 ] 5 0.001% 100.150% ** Level 4 read latency histogram (micros): Count: 724437 Average: 32.5945 StdDev: 338.65 Min: 1 Median: 2.2086 Max: 63985 Percentiles: P50: 2.21 P75: 3.32 P99: 538.94 P99.9: 1664.18 P99.99: 2508.94 ------------------------------------------------------ [ 0, 1 ] 7395 1.021% 1.021% ( 1, 2 ] 313154 43.227% 44.248% ######### ( 2, 3 ] 199711 27.568% 71.816% ###### ( 3, 4 ] 71763 9.906% 81.722% ## ( 4, 6 ] 71839 9.917% 91.638% ## ( 6, 10 ] 19021 2.626% 94.264% # ( 10, 15 ] 3644 0.503% 94.767% ( 15, 22 ] 429 0.059% 94.826% ( 22, 34 ] 80 0.011% 94.837% ( 34, 51 ] 160 0.022% 94.859% ( 51, 76 ] 109 0.015% 94.874% ( 76, 110 ] 253 0.035% 94.909% ( 110, 170 ] 5809 0.802% 95.711% ( 170, 250 ] 13902 1.919% 97.630% ( 250, 380 ] 5916 0.817% 98.447% ( 380, 580 ] 5043 0.696% 99.143% ( 580, 870 ] 3614 0.499% 99.642% ( 870, 1300 ] 1490 0.206% 99.847% ( 1300, 1900 ] 627 0.087% 99.934% ( 1900, 2900 ] 666 0.092% 100.026% ( 2900, 4400 ] 804 0.111% 100.137% ( 4400, 6600 ] 259 0.036% 100.173% ( 6600, 9900 ] 109 0.015% 100.188% ( 9900, 14000 ] 21 0.003% 100.191% ( 14000, 22000 ] 49 0.007% 100.197% ( 22000, 33000 ] 15 0.002% 100.199% ( 33000, 50000 ] 12 0.002% 100.201% ( 50000, 75000 ] 1 0.000% 100.201% ** Level 5 read latency histogram (micros): Count: 27580 Average: 483.1621 StdDev: 1089.19 Min: 90 Median: 240.9625 Max: 43963 Percentiles: P50: 240.96 P75: 456.44 P99: 4123.08 P99.9: 12570.13 P99.99: 42185.67 ------------------------------------------------------ ( 76, 110 ] 168 0.609% 0.609% ( 110, 170 ] 4286 15.540% 16.149% ### ( 170, 250 ] 10525 38.162% 54.311% ######## ( 250, 380 ] 4410 15.990% 70.301% ### ( 380, 580 ] 3391 12.295% 82.596% ## ( 580, 870 ] 2334 8.463% 91.059% ## ( 870, 1300 ] 1032 3.742% 94.801% # ( 1300, 1900 ] 382 1.385% 96.186% ( 1900, 2900 ] 384 1.392% 97.578% ( 2900, 4400 ] 481 1.744% 99.322% ( 4400, 6600 ] 108 0.392% 99.714% ( 6600, 9900 ] 41 0.149% 99.862% ( 9900, 14000 ] 16 0.058% 99.920% ( 14000, 22000 ] 10 0.036% 99.956% ( 22000, 33000 ] 6 0.022% 99.978% ( 33000, 50000 ] 6 0.022% 100.000% ** DB Stats ** Uptime(secs): 10.8 total, 3.8 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 181.49 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.070 H:M:S, 19.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 260.85 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-23:02:49 ... thread 0: (1000000,3000000) ops and (227981.6,198058.5) ops/second in (4.386318,15.147039) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 20/9 3.27 GB 262.8 2.5 0.0 2.5 5.3 2.8 0.0 1.9 99.9 214.2 25.18 18.67 51 0.494 2554K 13K 0.0 0.0 L1 7/7 389.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 957.67 MB 1.0 2.2 1.5 0.7 2.2 1.5 0.0 1.4 106.0 104.2 21.40 12.48 11 1.945 1558K 8260 0.0 0.0 L4 141/0 9.66 GB 1.0 4.7 2.0 2.7 4.3 1.6 0.2 2.1 132.7 120.8 36.62 23.44 19 1.928 1091K 173 0.0 0.0 L5 152/0 11.63 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 339/16 25.87 GB 0.0 9.4 3.5 5.9 11.8 5.9 2.6 4.2 115.9 144.8 83.20 54.59 81 1.027 5204K 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 9.4 3.5 5.9 11.8 5.9 2.6 4.2 115.9 144.8 83.20 54.59 81 1.027 5204K 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.4 3.5 5.9 8.9 3.1 0.0 0.0 137.0 130.0 70.41 46.24 39 1.805 5204K 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 226.1 12.80 8.35 42 0.305 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 15.2 total, 15.2 interval Flush(GB): cumulative 2.826, interval 2.826 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.77 GB write, 791.04 MB/s write, 9.42 GB read, 633.09 MB/s read, 83.2 seconds Interval compaction: 11.77 GB write, 795.16 MB/s write, 9.42 GB read, 636.39 MB/s read, 83.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 29, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 29, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 819421 Average: 28.6541 StdDev: 267.01 Min: 0 Median: 1.6108 Max: 71980 Percentiles: P50: 1.61 P75: 2.27 P99: 453.60 P99.9: 1558.90 P99.99: 3222.66 ------------------------------------------------------ [ 0, 1 ] 131172 16.008% 16.008% ### ( 1, 2 ] 456044 55.654% 71.662% ########### ( 2, 3 ] 100863 12.309% 83.971% ## ( 3, 4 ] 33387 4.074% 88.046% # ( 4, 6 ] 26174 3.194% 91.240% # ( 6, 10 ] 8033 0.980% 92.220% ( 10, 15 ] 1313 0.160% 92.381% ( 15, 22 ] 267 0.033% 92.413% ( 22, 34 ] 67 0.008% 92.421% ( 34, 51 ] 40 0.005% 92.426% ( 51, 76 ] 16 0.002% 92.428% ( 76, 110 ] 636 0.078% 92.506% ( 110, 170 ] 14044 1.714% 94.220% ( 170, 250 ] 29252 3.570% 97.790% # ( 250, 380 ] 8058 0.983% 98.773% ( 380, 580 ] 5062 0.618% 99.391% ( 580, 870 ] 2934 0.358% 99.749% ( 870, 1300 ] 1072 0.131% 99.880% ( 1300, 1900 ] 393 0.048% 99.928% ( 1900, 2900 ] 405 0.049% 99.977% ( 2900, 4400 ] 507 0.062% 100.039% ( 4400, 6600 ] 142 0.017% 100.056% ( 6600, 9900 ] 60 0.007% 100.063% ( 9900, 14000 ] 17 0.002% 100.066% ( 14000, 22000 ] 27 0.003% 100.069% ( 22000, 33000 ] 7 0.001% 100.070% ( 33000, 50000 ] 4 0.000% 100.070% ( 50000, 75000 ] 3 0.000% 100.071% ** Level 1 read latency histogram (micros): Count: 2816 Average: 361.6058 StdDev: 652.81 Min: 96 Median: 216.3568 Max: 8552 Percentiles: P50: 216.36 P75: 305.49 P99: 3811.61 P99.9: 8552.00 P99.99: 8552.00 ------------------------------------------------------ ( 76, 110 ] 27 0.959% 0.959% ( 110, 170 ] 608 21.591% 22.550% #### ( 170, 250 ] 1334 47.372% 69.922% ######### ( 250, 380 ] 335 11.896% 81.818% ## ( 380, 580 ] 233 8.274% 90.092% ## ( 580, 870 ] 140 4.972% 95.064% # ( 870, 1300 ] 56 1.989% 97.053% ( 1300, 1900 ] 19 0.675% 97.727% ( 1900, 2900 ] 17 0.604% 98.331% ( 2900, 4400 ] 31 1.101% 99.432% ( 4400, 6600 ] 6 0.213% 99.645% ( 6600, 9900 ] 10 0.355% 100.000% ** Level 2 read latency histogram (micros): Count: 294461 Average: 16.4339 StdDev: 288.85 Min: 1 Median: 2.1773 Max: 38942 Percentiles: P50: 2.18 P75: 3.09 P99: 13.53 P99.9: 2638.30 P99.99: 4264.27 ------------------------------------------------------ [ 0, 1 ] 3434 1.166% 1.166% ( 1, 2 ] 128535 43.651% 44.817% ######### ( 2, 3 ] 86078 29.232% 74.050% ###### ( 3, 4 ] 30530 10.368% 84.418% ## ( 4, 6 ] 32726 11.114% 95.531% ## ( 6, 10 ] 8537 2.899% 98.431% # ( 10, 15 ] 2377 0.807% 99.238% ( 15, 22 ] 308 0.105% 99.343% ( 22, 34 ] 34 0.012% 99.354% ( 34, 51 ] 65 0.022% 99.376% ( 51, 76 ] 68 0.023% 99.399% ( 76, 110 ] 17 0.006% 99.405% ( 110, 170 ] 20 0.007% 99.412% ( 170, 250 ] 52 0.018% 99.429% ( 250, 380 ] 187 0.064% 99.493% ( 380, 580 ] 376 0.128% 99.621% ( 580, 870 ] 349 0.119% 99.739% ( 870, 1300 ] 197 0.067% 99.806% ( 1300, 1900 ] 137 0.047% 99.853% ( 1900, 2900 ] 189 0.064% 99.917% ( 2900, 4400 ] 237 0.080% 99.997% ( 4400, 6600 ] 92 0.031% 100.029% ( 6600, 9900 ] 72 0.024% 100.053% ( 9900, 14000 ] 16 0.005% 100.058% ( 14000, 22000 ] 12 0.004% 100.062% ( 22000, 33000 ] 8 0.003% 100.065% ( 33000, 50000 ] 2 0.001% 100.066% ** Level 3 read latency histogram (micros): Count: 563543 Average: 18.1928 StdDev: 303.73 Min: 1 Median: 2.2030 Max: 39019 Percentiles: P50: 2.20 P75: 3.13 P99: 227.66 P99.9: 1057.69 P99.99: 1894.19 ------------------------------------------------------ [ 0, 1 ] 5580 0.990% 0.990% ( 1, 2 ] 242242 42.986% 43.976% ######### ( 2, 3 ] 167265 29.681% 73.657% ###### ( 3, 4 ] 58072 10.305% 83.961% ## ( 4, 6 ] 60404 10.719% 94.680% ## ( 6, 10 ] 15406 2.734% 97.414% # ( 10, 15 ] 3185 0.565% 97.979% ( 15, 22 ] 362 0.064% 98.043% ( 22, 34 ] 62 0.011% 98.054% ( 34, 51 ] 170 0.030% 98.084% ( 51, 76 ] 107 0.019% 98.103% ( 76, 110 ] 122 0.022% 98.125% ( 110, 170 ] 1994 0.354% 98.479% ( 170, 250 ] 4074 0.723% 99.202% ( 250, 380 ] 1349 0.239% 99.441% ( 380, 580 ] 1316 0.234% 99.675% ( 580, 870 ] 1063 0.189% 99.863% ( 870, 1300 ] 473 0.084% 99.947% ( 1300, 1900 ] 243 0.043% 99.990% ( 1900, 2900 ] 272 0.048% 100.039% ( 2900, 4400 ] 332 0.059% 100.098% ( 4400, 6600 ] 99 0.018% 100.115% ( 6600, 9900 ] 97 0.017% 100.132% ( 9900, 14000 ] 27 0.005% 100.137% ( 14000, 22000 ] 41 0.007% 100.144% ( 22000, 33000 ] 20 0.004% 100.148% ( 33000, 50000 ] 5 0.001% 100.149% ** Level 4 read latency histogram (micros): Count: 756808 Average: 39.6566 StdDev: 333.30 Min: 1 Median: 2.2896 Max: 63985 Percentiles: P50: 2.29 P75: 3.66 P99: 536.86 P99.9: 1634.58 P99.99: 2505.58 ------------------------------------------------------ [ 0, 1 ] 7395 0.977% 0.977% ( 1, 2 ] 313164 41.380% 42.357% ######## ( 2, 3 ] 199737 26.392% 68.749% ##### ( 3, 4 ] 71785 9.485% 78.234% ## ( 4, 6 ] 71889 9.499% 87.733% ## ( 6, 10 ] 19031 2.515% 90.248% # ( 10, 15 ] 3649 0.482% 90.730% ( 15, 22 ] 430 0.057% 90.787% ( 22, 34 ] 80 0.011% 90.797% ( 34, 51 ] 161 0.021% 90.818% ( 51, 76 ] 109 0.014% 90.833% ( 76, 110 ] 697 0.092% 90.925% ( 110, 170 ] 15881 2.098% 93.023% ( 170, 250 ] 31992 4.227% 97.251% # ( 250, 380 ] 8681 1.147% 98.398% ( 380, 580 ] 5814 0.768% 99.166% ( 580, 870 ] 3710 0.490% 99.656% ( 870, 1300 ] 1497 0.198% 99.854% ( 1300, 1900 ] 628 0.083% 99.937% ( 1900, 2900 ] 666 0.088% 100.025% ( 2900, 4400 ] 804 0.106% 100.131% ( 4400, 6600 ] 259 0.034% 100.165% ( 6600, 9900 ] 109 0.014% 100.180% ( 9900, 14000 ] 21 0.003% 100.182% ( 14000, 22000 ] 49 0.006% 100.189% ( 22000, 33000 ] 15 0.002% 100.191% ( 33000, 50000 ] 12 0.002% 100.193% ( 50000, 75000 ] 1 0.000% 100.193% ** Level 5 read latency histogram (micros): Count: 51021 Average: 352.2611 StdDev: 814.54 Min: 89 Median: 215.2575 Max: 43963 Percentiles: P50: 215.26 P75: 299.41 P99: 3392.07 P99.9: 8851.97 P99.99: 35544.05 ------------------------------------------------------ ( 76, 110 ] 465 0.911% 0.911% ( 110, 170 ] 11591 22.718% 23.629% ##### ( 170, 250 ] 23783 46.614% 70.244% ######### ( 250, 380 ] 6385 12.514% 82.758% ### ( 380, 580 ] 3923 7.689% 90.447% ## ( 580, 870 ] 2401 4.706% 95.153% # ( 870, 1300 ] 1038 2.034% 97.187% ( 1300, 1900 ] 382 0.749% 97.936% ( 1900, 2900 ] 385 0.755% 98.691% ( 2900, 4400 ] 481 0.943% 99.633% ( 4400, 6600 ] 108 0.212% 99.845% ( 6600, 9900 ] 41 0.080% 99.926% ( 9900, 14000 ] 16 0.031% 99.957% ( 14000, 22000 ] 10 0.020% 99.976% ( 22000, 33000 ] 6 0.012% 99.988% ( 33000, 50000 ] 6 0.012% 100.000% ** DB Stats ** Uptime(secs): 15.2 total, 4.4 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 193.84 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.751 H:M:S, 18.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 224.37 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.682 H:M:S, 15.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 352.703 micros/op 11296 ops/sec 20.183 seconds 227996 operations; 10.9 MB/s (55999 of 55999 found) Microseconds per read: Count: 227996 Average: 352.7141 StdDev: 1003.00 Min: 2 Median: 232.2161 Max: 97388 Percentiles: P50: 232.22 P75: 341.13 P99: 2601.00 P99.9: 14909.33 P99.99: 39669.49 ------------------------------------------------------ ( 1, 2 ] 10 0.004% 0.004% ( 2, 3 ] 60 0.026% 0.031% ( 3, 4 ] 87 0.038% 0.069% ( 4, 6 ] 192 0.084% 0.153% ( 6, 10 ] 360 0.158% 0.311% ( 10, 15 ] 1921 0.843% 1.154% ( 15, 22 ] 2956 1.297% 2.450% ( 22, 34 ] 4675 2.050% 4.501% ( 34, 51 ] 2028 0.889% 5.390% ( 51, 76 ] 411 0.180% 5.570% ( 76, 110 ] 159 0.070% 5.640% ( 110, 170 ] 18383 8.063% 13.703% ## ( 170, 250 ] 106411 46.672% 60.375% ######### ( 250, 380 ] 47566 20.863% 81.238% #### ( 380, 580 ] 24942 10.940% 92.177% ## ( 580, 870 ] 10162 4.457% 96.635% # ( 870, 1300 ] 3634 1.594% 98.228% ( 1300, 1900 ] 1173 0.514% 98.743% ( 1900, 2900 ] 836 0.367% 99.110% ( 2900, 4400 ] 1099 0.482% 99.592% ( 4400, 6600 ] 374 0.164% 99.756% ( 6600, 9900 ] 206 0.090% 99.846% ( 9900, 14000 ] 108 0.047% 99.893% ( 14000, 22000 ] 132 0.058% 99.951% ( 22000, 33000 ] 78 0.034% 99.986% ( 33000, 50000 ] 26 0.011% 99.997% ( 50000, 75000 ] 6 0.003% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3686376 rocksdb.bloom.filter.full.positive COUNT : 262788 rocksdb.bloom.filter.full.true.positive COUNT : 227063 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 933 rocksdb.memtable.miss COUNT : 227063 rocksdb.l0.hit COUNT : 29362 rocksdb.l1.hit COUNT : 4022 rocksdb.l2andup.hit COUNT : 193679 rocksdb.compaction.key.drop.new COUNT : 514690 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3574285 rocksdb.number.keys.read COUNT : 227996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3688662120 rocksdb.bytes.read COUNT : 227996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 479 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 5518626 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 11869885920 rocksdb.compact.write.bytes COUNT : 11337187482 rocksdb.flush.write.bytes COUNT : 3627581978 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 24 rocksdb.number.superversion_acquires COUNT : 651 rocksdb.number.superversion_releases COUNT : 623 rocksdb.number.superversion_cleanups COUNT : 158 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3603868672 rocksdb.memtable.garbage.bytes.at.flush COUNT : 6467584 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 12012690888 rocksdb.non.last.level.read.count COUNT : 2872805 rocksdb.block.checksum.compute.count COUNT : 3141589 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 231.553082 P95 : 762.772579 P99 : 2599.808612 P100 : 97387.000000 COUNT : 227996 SUM : 80221970 rocksdb.db.write.micros P50 : 1.582066 P95 : 5.086891 P99 : 7.853198 P100 : 557389.000000 COUNT : 3574285 SUM : 15397130 rocksdb.compaction.times.micros P50 : 1694117.647059 P95 : 2892857.142857 P99 : 3187575.000000 P100 : 3187575.000000 COUNT : 42 SUM : 73325627 rocksdb.compaction.times.cpu_micros P50 : 1132000.000000 P95 : 1818333.333333 P99 : 1854059.000000 P100 : 1854059.000000 COUNT : 42 SUM : 49093368 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 757.222222 P95 : 8195.000000 P99 : 11909.000000 P100 : 12697.000000 COUNT : 51 SUM : 80958 rocksdb.compaction.outfile.sync.micros P50 : 2366.666667 P95 : 7810.000000 P99 : 9775.333333 P100 : 12040.000000 COUNT : 134 SUM : 381727 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 174.210526 P95 : 861.714286 P99 : 4370.000000 P100 : 4746.000000 COUNT : 104 SUM : 38929 rocksdb.table.open.io.micros P50 : 5998.666667 P95 : 18675.000000 P99 : 21868.333333 P100 : 27423.000000 COUNT : 479 SUM : 3224739 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.195085 P95 : 9.993667 P99 : 241.765079 P100 : 67942.000000 COUNT : 2875327 SUM : 51295523 rocksdb.read.block.get.micros P50 : 201.084876 P95 : 610.481332 P99 : 2191.592292 P100 : 71984.000000 COUNT : 265849 SUM : 71258071 rocksdb.write.raw.block.micros P50 : 0.536598 P95 : 1.403093 P99 : 4.166395 P100 : 89683.000000 COUNT : 3625679 SUM : 14364280 rocksdb.numfiles.in.singlecompaction P50 : 3.095238 P95 : 5.766667 P99 : 12.000000 P100 : 12.000000 COUNT : 54 SUM : 195 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.561756 P95 : 1134.962148 P99 : 2185.521978 P100 : 557193.000000 COUNT : 26807 SUM : 5520931 rocksdb.sst.read.micros P50 : 1.983841 P95 : 213.791384 P99 : 515.631891 P100 : 71980.000000 COUNT : 2872951 SUM : 110109906 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 227996 SUM : 227996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3574285 SUM : 3688662120 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 218205.128205 P95 : 770100.000000 P99 : 882580.000000 P100 : 882580.000000 COUNT : 51 SUM : 14606800 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 145 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0