[2024-07-15 14:04:04.895694] Starting SPDK v24.01.1-pre git sha1 4b94202c6 / DPDK 23.11.0 initialization... [2024-07-15 14:04:04.895812] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid749943 ] Set seed to 1721045044893046 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-07-15 14:04:05.025540] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-07-15 14:04:05.068993] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-07-15 14:04:05.069046] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 749943' to capture a snapshot of events at runtime. [2024-07-15 14:04:05.069062] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid749943 for offline analysis/debug. [2024-07-15 14:04:05.069091] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Jul 15 14:04:09 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/07/15-14:04:17 ... thread 0: (1000000,1000000) ops and (121919.6,121919.6) ops/second in (8.202126,8.202126) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/10 2.67 GB 63.0 0.3 0.0 0.3 1.2 0.9 0.0 1.4 52.6 199.8 5.99 3.81 13 0.461 319K 368 0.0 0.0 L1 12/12 1.45 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 99.27 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 940.26 MB 0.9 1.9 1.1 0.8 1.9 1.1 0.0 1.7 108.6 107.5 17.85 9.52 11 1.623 201K 0 0.0 0.0 L4 173/0 9.60 GB 1.0 4.2 2.5 1.7 4.2 2.4 0.0 1.7 116.4 114.8 37.10 20.73 26 1.427 1767K 0 0.0 0.0 L5 1120/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1339/22 106.71 GB 0.0 6.4 3.6 2.8 7.2 4.4 2.4 8.4 107.8 121.0 60.94 34.07 50 1.219 2288K 368 0.0 0.0 Int 0/0 0.00 KB 0.0 6.4 3.6 2.8 7.2 4.4 2.4 8.4 107.8 121.0 60.94 34.07 50 1.219 2288K 368 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.4 3.6 2.8 6.3 3.5 0.0 0.0 116.7 115.3 56.29 31.50 38 1.481 2288K 368 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 189.8 4.65 2.57 12 0.387 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.4 total, 8.2 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.20 GB write, 882.03 MB/s write, 6.42 GB read, 785.71 MB/s read, 60.9 seconds Interval compaction: 7.20 GB write, 899.01 MB/s write, 6.42 GB read, 800.83 MB/s read, 60.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 21, pending-compaction-bytes-stops: 0, total-delays: 21, total-stops: 11, interval: 31 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 360988 Average: 14.4154 StdDev: 153.27 Min: 1 Median: 1.8334 Max: 15669 Percentiles: P50: 1.83 P75: 2.89 P99: 234.08 P99.9: 1804.52 P99.99: 3808.56 ------------------------------------------------------ [ 0, 1 ] 34293 9.500% 9.500% ## ( 1, 2 ] 175423 48.595% 58.095% ########## ( 2, 3 ] 68262 18.910% 77.005% #### ( 3, 4 ] 31726 8.789% 85.793% ## ( 4, 6 ] 30957 8.576% 94.369% ## ( 6, 10 ] 8840 2.449% 96.818% ( 10, 15 ] 2690 0.745% 97.563% ( 15, 22 ] 481 0.133% 97.696% ( 22, 34 ] 103 0.029% 97.725% ( 34, 51 ] 26 0.007% 97.732% ( 51, 76 ] 15 0.004% 97.736% ( 76, 110 ] 618 0.171% 97.907% ( 110, 170 ] 2730 0.756% 98.664% ( 170, 250 ] 1517 0.420% 99.084% ( 250, 380 ] 899 0.249% 99.333% ( 380, 580 ] 821 0.227% 99.560% ( 580, 870 ] 610 0.169% 99.729% ( 870, 1300 ] 411 0.114% 99.843% ( 1300, 1900 ] 245 0.068% 99.911% ( 1900, 2900 ] 169 0.047% 99.958% ( 2900, 4400 ] 193 0.053% 100.011% ( 4400, 6600 ] 47 0.013% 100.024% ( 6600, 9900 ] 15 0.004% 100.029% ( 9900, 14000 ] 6 0.002% 100.030% ( 14000, 22000 ] 2 0.001% 100.031% ** Level 1 read latency histogram (micros): Count: 487 Average: 684.9836 StdDev: 1564.33 Min: 56 Median: 223.1818 Max: 13278 Percentiles: P50: 223.18 P75: 565.58 P99: 10672.17 P99.9: 13278.00 P99.99: 13278.00 ------------------------------------------------------ ( 51, 76 ] 2 0.411% 0.411% ( 76, 110 ] 29 5.955% 6.366% # ( 110, 170 ] 154 31.622% 37.988% ###### ( 170, 250 ] 88 18.070% 56.057% #### ( 250, 380 ] 44 9.035% 65.092% ## ( 380, 580 ] 52 10.678% 75.770% ## ( 580, 870 ] 40 8.214% 83.984% ## ( 870, 1300 ] 28 5.749% 89.733% # ( 1300, 1900 ] 20 4.107% 93.840% # ( 1900, 2900 ] 7 1.437% 95.277% ( 2900, 4400 ] 12 2.464% 97.741% ( 4400, 6600 ] 2 0.411% 98.152% ( 6600, 9900 ] 3 0.616% 98.768% ( 9900, 14000 ] 6 1.232% 100.000% ** Level 2 read latency histogram (micros): Count: 50362 Average: 7.1434 StdDev: 82.61 Min: 1 Median: 1.9046 Max: 5560 Percentiles: P50: 1.90 P75: 3.06 P99: 12.87 P99.9: 1428.48 P99.99: 2898.28 ------------------------------------------------------ [ 0, 1 ] 2450 4.865% 4.865% # ( 1, 2 ] 25128 49.895% 54.760% ########## ( 2, 3 ] 9881 19.620% 74.379% #### ( 3, 4 ] 5375 10.673% 85.052% ## ( 4, 6 ] 5321 10.566% 95.618% ## ( 6, 10 ] 1430 2.839% 98.457% # ( 10, 15 ] 477 0.947% 99.404% ( 15, 22 ] 68 0.135% 99.539% ( 22, 34 ] 11 0.022% 99.561% ( 34, 51 ] 4 0.008% 99.569% ( 51, 76 ] 2 0.004% 99.573% ( 76, 110 ] 8 0.016% 99.589% ( 110, 170 ] 26 0.052% 99.641% ( 170, 250 ] 11 0.022% 99.662% ( 250, 380 ] 13 0.026% 99.688% ( 380, 580 ] 19 0.038% 99.726% ( 580, 870 ] 35 0.069% 99.795% ( 870, 1300 ] 46 0.091% 99.887% ( 1300, 1900 ] 31 0.062% 99.948% ( 1900, 2900 ] 21 0.042% 99.990% ( 2900, 4400 ] 4 0.008% 99.998% ( 4400, 6600 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 752418 Average: 11.7740 StdDev: 176.31 Min: 1 Median: 1.8870 Max: 45056 Percentiles: P50: 1.89 P75: 2.96 P99: 12.00 P99.9: 926.55 P99.99: 1299.87 ------------------------------------------------------ [ 0, 1 ] 33579 4.463% 4.463% # ( 1, 2 ] 386261 51.336% 55.799% ########## ( 2, 3 ] 151188 20.094% 75.892% #### ( 3, 4 ] 77581 10.311% 86.203% ## ( 4, 6 ] 76309 10.142% 96.345% ## ( 6, 10 ] 17369 2.308% 98.654% ( 10, 15 ] 6505 0.865% 99.518% ( 15, 22 ] 908 0.121% 99.639% ( 22, 34 ] 90 0.012% 99.651% ( 34, 51 ] 54 0.007% 99.658% ( 51, 76 ] 64 0.009% 99.666% ( 76, 110 ] 78 0.010% 99.677% ( 110, 170 ] 184 0.024% 99.701% ( 170, 250 ] 158 0.021% 99.722% ( 250, 380 ] 196 0.026% 99.748% ( 380, 580 ] 410 0.054% 99.803% ( 580, 870 ] 629 0.084% 99.886% ( 870, 1300 ] 780 0.104% 99.990% ( 1300, 1900 ] 638 0.085% 100.075% ( 1900, 2900 ] 436 0.058% 100.133% ( 2900, 4400 ] 494 0.066% 100.198% ( 4400, 6600 ] 138 0.018% 100.217% ( 6600, 9900 ] 34 0.005% 100.221% ( 9900, 14000 ] 4 0.001% 100.222% ( 14000, 22000 ] 13 0.002% 100.224% ( 33000, 50000 ] 1 0.000% 100.224% ** Level 4 read latency histogram (micros): Count: 446189 Average: 22.7314 StdDev: 227.01 Min: 1 Median: 1.8877 Max: 15776 Percentiles: P50: 1.89 P75: 2.99 P99: 424.64 P99.9: 2552.15 P99.99: 3389.78 ------------------------------------------------------ [ 0, 1 ] 19880 4.456% 4.456% # ( 1, 2 ] 228935 51.309% 55.764% ########## ( 2, 3 ] 87082 19.517% 75.281% #### ( 3, 4 ] 45397 10.174% 85.456% ## ( 4, 6 ] 42964 9.629% 95.085% ## ( 6, 10 ] 10122 2.269% 97.353% ( 10, 15 ] 3977 0.891% 98.245% ( 15, 22 ] 548 0.123% 98.368% ( 22, 34 ] 57 0.013% 98.380% ( 34, 51 ] 41 0.009% 98.389% ( 51, 76 ] 28 0.006% 98.396% ( 76, 110 ] 247 0.055% 98.451% ( 110, 170 ] 1086 0.243% 98.694% ( 170, 250 ] 671 0.150% 98.845% ( 250, 380 ] 518 0.116% 98.961% ( 380, 580 ] 780 0.175% 99.136% ( 580, 870 ] 1049 0.235% 99.371% ( 870, 1300 ] 1133 0.254% 99.625% ( 1300, 1900 ] 830 0.186% 99.811% ( 1900, 2900 ] 610 0.137% 99.948% ( 2900, 4400 ] 580 0.130% 100.078% ( 4400, 6600 ] 168 0.038% 100.115% ( 6600, 9900 ] 35 0.008% 100.123% ( 9900, 14000 ] 7 0.002% 100.125% ( 14000, 22000 ] 2 0.000% 100.125% ** Level 5 read latency histogram (micros): Count: 29949 Average: 771.8171 StdDev: 1314.39 Min: 78 Median: 246.4030 Max: 68906 Percentiles: P50: 246.40 P75: 800.03 P99: 6015.82 P99.9: 9298.95 P99.99: 11724.55 ------------------------------------------------------ ( 76, 110 ] 1948 6.504% 6.504% # ( 110, 170 ] 8704 29.063% 35.567% ###### ( 170, 250 ] 4526 15.112% 50.679% ### ( 250, 380 ] 2625 8.765% 59.444% ## ( 380, 580 ] 2831 9.453% 68.897% ## ( 580, 870 ] 2409 8.044% 76.941% ## ( 870, 1300 ] 2103 7.022% 83.963% # ( 1300, 1900 ] 1447 4.832% 88.794% # ( 1900, 2900 ] 1201 4.010% 92.804% # ( 2900, 4400 ] 1395 4.658% 97.462% # ( 4400, 6600 ] 627 2.094% 99.556% ( 6600, 9900 ] 126 0.421% 99.977% ( 9900, 14000 ] 9 0.030% 100.007% ( 14000, 22000 ] 2 0.007% 100.013% ( 33000, 50000 ] 3 0.010% 100.023% ( 50000, 75000 ] 1 0.003% 100.027% ** DB Stats ** Uptime(secs): 8.4 total, 8.2 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 117.70 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.067 H:M:S, 48.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 119.95 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.067 H:M:S, 49.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/07/15-14:04:38 ... thread 0: (1000000,2000000) ops and (47794.5,68669.4) ops/second in (20.922913,29.125039) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 14/12 3.72 GB 44.1 1.8 0.0 1.8 3.7 1.9 0.0 1.9 120.6 249.4 15.15 12.39 36 0.421 1855K 1785 0.0 0.0 L1 12/12 1.45 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 99.27 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 940.26 MB 0.9 1.9 1.1 0.8 1.9 1.1 0.0 1.7 108.6 107.5 17.85 9.52 11 1.623 201K 0 0.0 0.0 L4 173/0 9.60 GB 1.0 4.2 2.5 1.7 4.2 2.4 0.0 1.7 116.4 114.8 37.10 20.73 26 1.427 1767K 0 0.0 0.0 L5 1120/0 91.97 GB 0.9 0.0 0.0 0.0 0.0 0.0 2.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1338/24 107.75 GB 0.0 7.9 3.6 4.3 9.7 5.5 2.4 5.1 115.3 142.0 70.10 42.65 73 0.960 3823K 1785 0.0 0.0 Int 0/0 0.00 KB 0.0 7.9 3.6 4.3 9.7 5.5 2.4 5.1 115.3 142.0 70.10 42.65 73 0.960 3823K 1785 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 7.9 3.6 4.3 7.8 3.5 0.0 0.0 130.2 128.9 62.08 37.17 44 1.411 3823K 1785 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 0.0 243.6 8.02 5.48 29 0.276 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 29.3 total, 29.1 interval Flush(GB): cumulative 1.907, interval 1.907 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.72 GB write, 340.00 MB/s write, 7.89 GB read, 275.98 MB/s read, 70.1 seconds Interval compaction: 9.72 GB write, 341.84 MB/s write, 7.89 GB read, 277.47 MB/s read, 70.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 28, pending-compaction-bytes-stops: 0, total-delays: 28, total-stops: 11, interval: 38 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1170493 Average: 21.2259 StdDev: 101.07 Min: 0 Median: 1.7057 Max: 15669 Percentiles: P50: 1.71 P75: 2.92 P99: 243.61 P99.9: 753.07 P99.99: 1294.16 ------------------------------------------------------ [ 0, 1 ] 208480 17.811% 17.811% #### ( 1, 2 ] 533905 45.614% 63.425% ######### ( 2, 3 ] 147124 12.569% 75.994% ### ( 3, 4 ] 78067 6.670% 82.664% # ( 4, 6 ] 59397 5.075% 87.738% # ( 6, 10 ] 14759 1.261% 88.999% ( 10, 15 ] 4274 0.365% 89.365% ( 15, 22 ] 1250 0.107% 89.471% ( 22, 34 ] 331 0.028% 89.500% ( 34, 51 ] 52 0.004% 89.504% ( 51, 76 ] 26 0.002% 89.506% ( 76, 110 ] 22509 1.923% 91.429% ( 110, 170 ] 73245 6.258% 97.687% # ( 170, 250 ] 16705 1.427% 99.114% ( 250, 380 ] 4744 0.405% 99.519% ( 380, 580 ] 3654 0.312% 99.832% ( 580, 870 ] 1343 0.115% 99.946% ( 870, 1300 ] 519 0.044% 99.991% ( 1300, 1900 ] 261 0.022% 100.013% ( 1900, 2900 ] 188 0.016% 100.029% ( 2900, 4400 ] 208 0.018% 100.047% ( 4400, 6600 ] 47 0.004% 100.051% ( 6600, 9900 ] 15 0.001% 100.052% ( 9900, 14000 ] 6 0.001% 100.053% ( 14000, 22000 ] 2 0.000% 100.053% ** Level 1 read latency histogram (micros): Count: 125462 Average: 13.4153 StdDev: 112.99 Min: 1 Median: 1.8050 Max: 13278 Percentiles: P50: 1.80 P75: 2.92 P99: 188.30 P99.9: 716.08 P99.99: 3956.72 ------------------------------------------------------ [ 0, 1 ] 8161 6.505% 6.505% # ( 1, 2 ] 67790 54.032% 60.537% ########### ( 2, 3 ] 19702 15.704% 76.241% ### ( 3, 4 ] 10143 8.085% 84.325% ## ( 4, 6 ] 8699 6.934% 91.259% # ( 6, 10 ] 2990 2.383% 93.642% ( 10, 15 ] 441 0.352% 93.993% ( 15, 22 ] 317 0.253% 94.246% ( 22, 34 ] 35 0.028% 94.274% ( 34, 51 ] 3 0.002% 94.276% ( 51, 76 ] 2 0.002% 94.278% ( 76, 110 ] 1415 1.128% 95.406% ( 110, 170 ] 4309 3.435% 98.840% # ( 170, 250 ] 876 0.698% 99.539% ( 250, 380 ] 221 0.176% 99.715% ( 380, 580 ] 195 0.155% 99.870% ( 580, 870 ] 80 0.064% 99.934% ( 870, 1300 ] 32 0.026% 99.959% ( 1300, 1900 ] 23 0.018% 99.978% ( 1900, 2900 ] 7 0.006% 99.983% ( 2900, 4400 ] 12 0.010% 99.993% ( 4400, 6600 ] 2 0.002% 99.994% ( 6600, 9900 ] 3 0.002% 99.997% ( 9900, 14000 ] 6 0.005% 100.002% ** Level 2 read latency histogram (micros): Count: 50951 Average: 8.7138 StdDev: 84.29 Min: 1 Median: 1.9163 Max: 5560 Percentiles: P50: 1.92 P75: 3.14 P99: 129.40 P99.9: 1436.43 P99.99: 2895.68 ------------------------------------------------------ [ 0, 1 ] 2450 4.809% 4.809% # ( 1, 2 ] 25128 49.318% 54.127% ########## ( 2, 3 ] 9881 19.393% 73.520% #### ( 3, 4 ] 5375 10.549% 84.069% ## ( 4, 6 ] 5321 10.443% 94.512% ## ( 6, 10 ] 1430 2.807% 97.319% # ( 10, 15 ] 477 0.936% 98.255% ( 15, 22 ] 68 0.133% 98.389% ( 22, 34 ] 11 0.022% 98.410% ( 34, 51 ] 4 0.008% 98.418% ( 51, 76 ] 2 0.004% 98.422% ( 76, 110 ] 171 0.336% 98.758% ( 110, 170 ] 382 0.750% 99.507% ( 170, 250 ] 57 0.112% 99.619% ( 250, 380 ] 27 0.053% 99.672% ( 380, 580 ] 27 0.053% 99.725% ( 580, 870 ] 36 0.071% 99.796% ( 870, 1300 ] 46 0.090% 99.886% ( 1300, 1900 ] 31 0.061% 99.947% ( 1900, 2900 ] 22 0.043% 99.990% ( 2900, 4400 ] 4 0.008% 99.998% ( 4400, 6600 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 758134 Average: 12.8482 StdDev: 176.27 Min: 1 Median: 1.8944 Max: 45056 Percentiles: P50: 1.89 P75: 2.98 P99: 94.10 P99.9: 927.99 P99.99: 1302.99 ------------------------------------------------------ [ 0, 1 ] 33579 4.429% 4.429% # ( 1, 2 ] 386261 50.949% 55.378% ########## ( 2, 3 ] 151188 19.942% 75.320% #### ( 3, 4 ] 77581 10.233% 85.553% ## ( 4, 6 ] 76309 10.065% 95.619% ## ( 6, 10 ] 17369 2.291% 97.910% ( 10, 15 ] 6505 0.858% 98.768% ( 15, 22 ] 908 0.120% 98.888% ( 22, 34 ] 90 0.012% 98.899% ( 34, 51 ] 54 0.007% 98.907% ( 51, 76 ] 64 0.008% 98.915% ( 76, 110 ] 1211 0.160% 99.075% ( 110, 170 ] 3706 0.489% 99.564% ( 170, 250 ] 899 0.119% 99.682% ( 250, 380 ] 372 0.049% 99.731% ( 380, 580 ] 517 0.068% 99.799% ( 580, 870 ] 657 0.087% 99.886% ( 870, 1300 ] 785 0.104% 99.990% ( 1300, 1900 ] 640 0.084% 100.074% ( 1900, 2900 ] 438 0.058% 100.132% ( 2900, 4400 ] 494 0.065% 100.197% ( 4400, 6600 ] 138 0.018% 100.215% ( 6600, 9900 ] 34 0.004% 100.220% ( 9900, 14000 ] 4 0.001% 100.220% ( 14000, 22000 ] 13 0.002% 100.222% ( 33000, 50000 ] 1 0.000% 100.222% ** Level 4 read latency histogram (micros): Count: 497168 Average: 34.9242 StdDev: 219.82 Min: 1 Median: 1.9974 Max: 15776 Percentiles: P50: 2.00 P75: 3.77 P99: 480.44 P99.9: 2459.79 P99.99: 3355.12 ------------------------------------------------------ [ 0, 1 ] 19880 3.999% 3.999% # ( 1, 2 ] 229291 46.119% 50.118% ######### ( 2, 3 ] 88246 17.750% 67.868% #### ( 3, 4 ] 46134 9.279% 77.147% ## ( 4, 6 ] 44046 8.859% 86.007% ## ( 6, 10 ] 10221 2.056% 88.062% ( 10, 15 ] 4002 0.805% 88.867% ( 15, 22 ] 556 0.112% 88.979% ( 22, 34 ] 60 0.012% 88.991% ( 34, 51 ] 41 0.008% 88.999% ( 51, 76 ] 36 0.007% 89.007% ( 76, 110 ] 10027 2.017% 91.024% ( 110, 170 ] 30558 6.146% 97.170% # ( 170, 250 ] 6338 1.275% 98.445% ( 250, 380 ] 1893 0.381% 98.826% ( 380, 580 ] 1727 0.347% 99.173% ( 580, 870 ] 1277 0.257% 99.430% ( 870, 1300 ] 1161 0.234% 99.663% ( 1300, 1900 ] 832 0.167% 99.831% ( 1900, 2900 ] 616 0.124% 99.955% ( 2900, 4400 ] 581 0.117% 100.071% ( 4400, 6600 ] 168 0.034% 100.105% ( 6600, 9900 ] 35 0.007% 100.112% ( 9900, 14000 ] 7 0.001% 100.114% ( 14000, 22000 ] 2 0.000% 100.114% ** Level 5 read latency histogram (micros): Count: 381739 Average: 200.6213 StdDev: 411.66 Min: 72 Median: 140.6668 Max: 68906 Percentiles: P50: 140.67 P75: 166.15 P99: 1671.29 P99.9: 5214.95 P99.99: 6420.44 ------------------------------------------------------ ( 51, 76 ] 47 0.012% 0.012% ( 76, 110 ] 75957 19.898% 19.910% #### ( 110, 170 ] 224736 58.872% 78.782% ############ ( 170, 250 ] 46681 12.229% 91.010% ## ( 250, 380 ] 13020 3.411% 94.421% # ( 380, 580 ] 10064 2.636% 97.057% # ( 580, 870 ] 4174 1.093% 98.151% ( 870, 1300 ] 2328 0.610% 98.760% ( 1300, 1900 ] 1478 0.387% 99.148% ( 1900, 2900 ] 1238 0.324% 99.472% ( 2900, 4400 ] 1402 0.367% 99.839% ( 4400, 6600 ] 627 0.164% 100.003% ( 6600, 9900 ] 126 0.033% 100.036% ( 9900, 14000 ] 9 0.002% 100.039% ( 14000, 22000 ] 2 0.001% 100.039% ( 33000, 50000 ] 3 0.001% 100.040% ( 50000, 75000 ] 1 0.000% 100.040% ** DB Stats ** Uptime(secs): 29.3 total, 20.9 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 67.22 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:20.836 H:M:S, 71.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 47.04 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:16.769 H:M:S, 80.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 2024/07/15-14:04:56 ... thread 0: (1000000,3000000) ops and (56533.6,64083.9) ops/second in (17.688587,46.813626) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 10/8 2.76 GB 56.6 3.8 0.0 3.8 6.6 2.8 0.0 2.4 143.4 247.1 27.23 22.71 52 0.524 3964K 12K 0.0 0.0 L1 16/16 1.90 GB 0.0 3.3 1.8 1.4 3.2 1.8 0.0 1.8 83.0 82.1 40.22 20.20 1 40.215 3389K 36K 0.0 0.0 L2 1/0 81.69 MB 0.8 0.0 0.0 0.0 0.0 0.0 1.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/0 925.61 MB 0.9 4.0 2.2 1.8 3.9 2.1 0.3 1.8 112.2 111.2 36.08 21.53 22 1.640 1835K 8994 0.0 0.0 L4 179/0 9.59 GB 1.0 7.2 3.8 3.4 7.1 3.7 0.0 1.9 128.7 126.5 57.55 35.29 46 1.251 3457K 0 0.0 0.0 L5 1136/0 93.26 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.6 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1362/24 108.48 GB 0.0 18.3 7.8 10.4 20.8 10.4 5.3 7.5 116.1 132.4 161.08 99.72 121 1.331 12M 57K 0.0 0.0 Int 0/0 0.00 KB 0.0 18.3 7.8 10.4 20.8 10.4 5.3 7.5 116.1 132.4 161.08 99.72 121 1.331 12M 57K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 18.3 7.8 10.4 18.1 7.6 0.0 0.0 125.5 124.1 148.99 91.66 79 1.886 12M 57K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 234.6 12.09 8.07 42 0.288 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 47.0 total, 46.8 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 20.82 GB write, 453.91 MB/s write, 18.26 GB read, 398.00 MB/s read, 161.1 seconds Interval compaction: 20.82 GB write, 455.44 MB/s write, 18.26 GB read, 399.34 MB/s read, 161.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 12, pending-compaction-bytes-delays: 53, pending-compaction-bytes-stops: 0, total-delays: 53, total-stops: 12, interval: 64 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1664914 Average: 25.2537 StdDev: 156.46 Min: 0 Median: 1.6939 Max: 45753 Percentiles: P50: 1.69 P75: 2.93 P99: 297.24 P99.9: 1028.21 P99.99: 1926.14 ------------------------------------------------------ [ 0, 1 ] 301016 18.080% 18.080% #### ( 1, 2 ] 765881 46.001% 64.081% ######### ( 2, 3 ] 196199 11.784% 75.866% ## ( 3, 4 ] 98926 5.942% 81.807% # ( 4, 6 ] 77403 4.649% 86.456% # ( 6, 10 ] 19136 1.149% 87.606% ( 10, 15 ] 5521 0.332% 87.937% ( 15, 22 ] 1413 0.085% 88.022% ( 22, 34 ] 353 0.021% 88.043% ( 34, 51 ] 74 0.004% 88.048% ( 51, 76 ] 53 0.003% 88.051% ( 76, 110 ] 37537 2.255% 90.306% ( 110, 170 ] 115404 6.932% 97.237% # ( 170, 250 ] 26365 1.584% 98.821% ( 250, 380 ] 8216 0.493% 99.314% ( 380, 580 ] 6409 0.385% 99.699% ( 580, 870 ] 2862 0.172% 99.871% ( 870, 1300 ] 1313 0.079% 99.950% ( 1300, 1900 ] 658 0.040% 99.989% ( 1900, 2900 ] 402 0.024% 100.014% ( 2900, 4400 ] 423 0.025% 100.039% ( 4400, 6600 ] 103 0.006% 100.045% ( 6600, 9900 ] 19 0.001% 100.046% ( 9900, 14000 ] 7 0.000% 100.047% ( 14000, 22000 ] 8 0.000% 100.047% ( 22000, 33000 ] 9 0.001% 100.048% ( 33000, 50000 ] 6 0.000% 100.048% ** Level 1 read latency histogram (micros): Count: 625463 Average: 10.1293 StdDev: 218.50 Min: 0 Median: 1.7314 Max: 49168 Percentiles: P50: 1.73 P75: 2.70 P99: 146.38 P99.9: 715.47 P99.99: 1295.41 ------------------------------------------------------ [ 0, 1 ] 62596 10.008% 10.008% ## ( 1, 2 ] 342010 54.681% 64.689% ########### ( 2, 3 ] 92571 14.800% 79.489% ### ( 3, 4 ] 49446 7.906% 87.395% ## ( 4, 6 ] 47024 7.518% 94.913% ## ( 6, 10 ] 14549 2.326% 97.239% ( 10, 15 ] 3412 0.546% 97.785% ( 15, 22 ] 993 0.159% 97.944% ( 22, 34 ] 97 0.016% 97.959% ( 34, 51 ] 58 0.009% 97.968% ( 51, 76 ] 50 0.008% 97.976% ( 76, 110 ] 2216 0.354% 98.331% ( 110, 170 ] 6904 1.104% 99.434% ( 170, 250 ] 1543 0.247% 99.681% ( 250, 380 ] 569 0.091% 99.772% ( 380, 580 ] 594 0.095% 99.867% ( 580, 870 ] 440 0.070% 99.937% ( 870, 1300 ] 332 0.053% 99.991% ( 1300, 1900 ] 142 0.023% 100.013% ( 1900, 2900 ] 84 0.013% 100.027% ( 2900, 4400 ] 118 0.019% 100.046% ( 4400, 6600 ] 31 0.005% 100.051% ( 6600, 9900 ] 17 0.003% 100.053% ( 9900, 14000 ] 15 0.002% 100.056% ( 14000, 22000 ] 12 0.002% 100.058% ( 22000, 33000 ] 11 0.002% 100.059% ( 33000, 50000 ] 7 0.001% 100.060% ** Level 2 read latency histogram (micros): Count: 76455 Average: 8.7202 StdDev: 120.49 Min: 1 Median: 1.9366 Max: 17732 Percentiles: P50: 1.94 P75: 3.21 P99: 128.43 P99.9: 1249.12 P99.99: 3488.62 ------------------------------------------------------ [ 0, 1 ] 3245 4.244% 4.244% # ( 1, 2 ] 37349 48.851% 53.095% ########## ( 2, 3 ] 15041 19.673% 72.768% #### ( 3, 4 ] 8195 10.719% 83.487% ## ( 4, 6 ] 8440 11.039% 94.526% ## ( 6, 10 ] 2171 2.840% 97.366% # ( 10, 15 ] 719 0.940% 98.306% ( 15, 22 ] 94 0.123% 98.429% ( 22, 34 ] 13 0.017% 98.446% ( 34, 51 ] 10 0.013% 98.459% ( 51, 76 ] 6 0.008% 98.467% ( 76, 110 ] 233 0.305% 98.772% ( 110, 170 ] 568 0.743% 99.515% ( 170, 250 ] 94 0.123% 99.638% ( 250, 380 ] 46 0.060% 99.698% ( 380, 580 ] 46 0.060% 99.758% ( 580, 870 ] 53 0.069% 99.827% ( 870, 1300 ] 63 0.082% 99.910% ( 1300, 1900 ] 32 0.042% 99.952% ( 1900, 2900 ] 27 0.035% 99.987% ( 2900, 4400 ] 6 0.008% 99.995% ( 4400, 6600 ] 2 0.003% 99.997% ( 14000, 22000 ] 2 0.003% 100.000% ** Level 3 read latency histogram (micros): Count: 1364658 Average: 10.9775 StdDev: 203.19 Min: 1 Median: 1.8952 Max: 48968 Percentiles: P50: 1.90 P75: 2.97 P99: 15.65 P99.9: 631.37 P99.99: 1057.61 ------------------------------------------------------ [ 0, 1 ] 56330 4.128% 4.128% # ( 1, 2 ] 699261 51.241% 55.369% ########## ( 2, 3 ] 274903 20.144% 75.513% #### ( 3, 4 ] 140800 10.318% 85.831% ## ( 4, 6 ] 140060 10.263% 96.094% ## ( 6, 10 ] 29505 2.162% 98.256% ( 10, 15 ] 10029 0.735% 98.991% ( 15, 22 ] 1329 0.097% 99.088% ( 22, 34 ] 128 0.009% 99.098% ( 34, 51 ] 174 0.013% 99.110% ( 51, 76 ] 142 0.010% 99.121% ( 76, 110 ] 1736 0.127% 99.248% ( 110, 170 ] 5717 0.419% 99.667% ( 170, 250 ] 1440 0.106% 99.773% ( 250, 380 ] 706 0.052% 99.824% ( 380, 580 ] 864 0.063% 99.888% ( 580, 870 ] 956 0.070% 99.958% ( 870, 1300 ] 1012 0.074% 100.032% ( 1300, 1900 ] 767 0.056% 100.088% ( 1900, 2900 ] 557 0.041% 100.129% ( 2900, 4400 ] 620 0.045% 100.174% ( 4400, 6600 ] 181 0.013% 100.188% ( 6600, 9900 ] 57 0.004% 100.192% ( 9900, 14000 ] 10 0.001% 100.192% ( 14000, 22000 ] 29 0.002% 100.195% ( 22000, 33000 ] 17 0.001% 100.196% ( 33000, 50000 ] 5 0.000% 100.196% ** Level 4 read latency histogram (micros): Count: 966491 Average: 29.9561 StdDev: 216.56 Min: 0 Median: 1.9888 Max: 39423 Percentiles: P50: 1.99 P75: 3.65 P99: 401.17 P99.9: 1832.04 P99.99: 2675.03 ------------------------------------------------------ [ 0, 1 ] 35817 3.706% 3.706% # ( 1, 2 ] 452519 46.821% 50.527% ######### ( 2, 3 ] 176584 18.271% 68.797% #### ( 3, 4 ] 92511 9.572% 78.369% ## ( 4, 6 ] 90647 9.379% 87.748% ## ( 6, 10 ] 20765 2.148% 89.897% ( 10, 15 ] 8207 0.849% 90.746% ( 15, 22 ] 974 0.101% 90.847% ( 22, 34 ] 90 0.009% 90.856% ( 34, 51 ] 111 0.011% 90.867% ( 51, 76 ] 89 0.009% 90.877% ( 76, 110 ] 15506 1.604% 92.481% ( 110, 170 ] 48537 5.022% 97.503% # ( 170, 250 ] 10571 1.094% 98.597% ( 250, 380 ] 3561 0.368% 98.965% ( 380, 580 ] 3184 0.329% 99.295% ( 580, 870 ] 2539 0.263% 99.557% ( 870, 1300 ] 2103 0.218% 99.775% ( 1300, 1900 ] 1364 0.141% 99.916% ( 1900, 2900 ] 923 0.096% 100.011% ( 2900, 4400 ] 870 0.090% 100.102% ( 4400, 6600 ] 233 0.024% 100.126% ( 6600, 9900 ] 41 0.004% 100.130% ( 9900, 14000 ] 8 0.001% 100.131% ( 14000, 22000 ] 13 0.001% 100.132% ( 22000, 33000 ] 5 0.001% 100.133% ( 33000, 50000 ] 2 0.000% 100.133% ** Level 5 read latency histogram (micros): Count: 596559 Average: 208.0306 StdDev: 472.09 Min: 72 Median: 142.0469 Max: 68906 Percentiles: P50: 142.05 P75: 167.82 P99: 1653.68 P99.9: 4947.76 P99.99: 6437.27 ------------------------------------------------------ ( 51, 76 ] 52 0.009% 0.009% ( 76, 110 ] 112756 18.901% 18.910% #### ( 110, 170 ] 347250 58.209% 77.119% ############ ( 170, 250 ] 74747 12.530% 89.648% ### ( 250, 380 ] 22309 3.740% 93.388% # ( 380, 580 ] 17583 2.947% 96.335% # ( 580, 870 ] 8915 1.494% 97.830% ( 870, 1300 ] 5302 0.889% 98.718% ( 1300, 1900 ] 2849 0.478% 99.196% ( 1900, 2900 ] 1964 0.329% 99.525% ( 2900, 4400 ] 2038 0.342% 99.867% ( 4400, 6600 ] 793 0.133% 100.000% ( 6600, 9900 ] 138 0.023% 100.023% ( 9900, 14000 ] 14 0.002% 100.025% ( 14000, 22000 ] 20 0.003% 100.029% ( 22000, 33000 ] 16 0.003% 100.031% ( 33000, 50000 ] 19 0.003% 100.035% ( 50000, 75000 ] 2 0.000% 100.035% ** DB Stats ** Uptime(secs): 47.0 total, 17.7 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 62.86 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:33.914 H:M:S, 72.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 55.64 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:13.078 H:M:S, 73.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 237.599 micros/op 16818 ops/sec 60.172 seconds 1011996 operations; 16.3 MB/s (250999 of 250999 found) Microseconds per read: Count: 1011996 Average: 237.6013 StdDev: 730.43 Min: 2 Median: 168.8594 Max: 118161 Percentiles: P50: 168.86 P75: 228.86 P99: 1144.27 P99.9: 11560.29 P99.99: 31048.06 ------------------------------------------------------ ( 1, 2 ] 3 0.000% 0.000% ( 2, 3 ] 46 0.005% 0.005% ( 3, 4 ] 68 0.007% 0.012% ( 4, 6 ] 82 0.008% 0.020% ( 6, 10 ] 336 0.033% 0.053% ( 10, 15 ] 4070 0.402% 0.455% ( 15, 22 ] 3671 0.363% 0.818% ( 22, 34 ] 7916 0.782% 1.600% ( 34, 51 ] 2847 0.281% 1.881% ( 51, 76 ] 350 0.035% 1.916% ( 76, 110 ] 24276 2.399% 4.315% ( 110, 170 ] 471292 46.571% 50.885% ######### ( 170, 250 ] 331675 32.774% 83.660% ####### ( 250, 380 ] 95701 9.457% 93.116% ## ( 380, 580 ] 39715 3.924% 97.041% # ( 580, 870 ] 15492 1.531% 98.572% ( 870, 1300 ] 6798 0.672% 99.243% ( 1300, 1900 ] 2891 0.286% 99.529% ( 1900, 2900 ] 1621 0.160% 99.689% ( 2900, 4400 ] 1342 0.133% 99.822% ( 4400, 6600 ] 435 0.043% 99.865% ( 6600, 9900 ] 193 0.019% 99.884% ( 9900, 14000 ] 405 0.040% 99.924% ( 14000, 22000 ] 502 0.050% 99.973% ( 22000, 33000 ] 204 0.020% 99.994% ( 33000, 50000 ] 57 0.006% 99.999% ( 50000, 75000 ] 5 0.000% 100.000% ( 75000, 110000 ] 2 0.000% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14064600 rocksdb.bloom.filter.full.positive COUNT : 1148306 rocksdb.bloom.filter.full.true.positive COUNT : 1011566 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 430 rocksdb.memtable.miss COUNT : 1011566 rocksdb.l0.hit COUNT : 35549 rocksdb.l1.hit COUNT : 16027 rocksdb.l2andup.hit COUNT : 959990 rocksdb.compaction.key.drop.new COUNT : 218152 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3385649 rocksdb.number.keys.read COUNT : 1011996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3493989768 rocksdb.bytes.read COUNT : 1011996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1633 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 45490377 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 21452872650 rocksdb.compact.write.bytes COUNT : 21224554665 rocksdb.flush.write.bytes COUNT : 3435132720 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 35 rocksdb.number.superversion_acquires COUNT : 802 rocksdb.number.superversion_releases COUNT : 775 rocksdb.number.superversion_cleanups COUNT : 200 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 168 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3407289344 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1193984 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 25345668696 rocksdb.non.last.level.read.count COUNT : 5982930 rocksdb.block.checksum.compute.count COUNT : 6353461 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 168.091588 P95 : 474.809668 P99 : 1143.241597 P100 : 118160.000000 COUNT : 1011996 SUM : 239646998 rocksdb.db.write.micros P50 : 1.681519 P95 : 5.758097 P99 : 154.123451 P100 : 630472.000000 COUNT : 3385649 SUM : 55541157 rocksdb.compaction.times.micros P50 : 1345121.951220 P95 : 2558333.333333 P99 : 36040000.000000 P100 : 40215036.000000 COUNT : 81 SUM : 150924771 rocksdb.compaction.times.cpu_micros P50 : 855606.060606 P95 : 1613000.000000 P99 : 15520000.000000 P100 : 20196252.000000 COUNT : 81 SUM : 93550688 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 785.000000 P95 : 13077.500000 P99 : 17042.000000 P100 : 17042.000000 COUNT : 49 SUM : 111010 rocksdb.compaction.outfile.sync.micros P50 : 9552.631579 P95 : 139000.000000 P99 : 243800.000000 P100 : 274173.000000 COUNT : 262 SUM : 6943421 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 278.437500 P95 : 4887.142857 P99 : 7408.000000 P100 : 7408.000000 COUNT : 149 SUM : 141114 rocksdb.table.open.io.micros P50 : 9004.875000 P95 : 20215.969582 P99 : 25668.500000 P100 : 90344.000000 COUNT : 1633 SUM : 15175085 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.945384 P95 : 13.027611 P99 : 186.379609 P100 : 49174.000000 COUNT : 5194862 SUM : 83013966 rocksdb.read.block.get.micros P50 : 140.810769 P95 : 363.615931 P99 : 1097.015888 P100 : 68966.000000 COUNT : 1157007 SUM : 205706205 rocksdb.write.raw.block.micros P50 : 0.548612 P95 : 1.599487 P99 : 4.439914 P100 : 45835.000000 COUNT : 5973149 SUM : 17177229 rocksdb.numfiles.in.singlecompaction P50 : 2.750000 P95 : 3.996053 P99 : 18.395000 P100 : 20.000000 COUNT : 103 SUM : 355 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.573062 P95 : 1228.510949 P99 : 2626.608449 P100 : 630268.000000 COUNT : 255472 SUM : 45514992 rocksdb.sst.read.micros P50 : 2.045239 P95 : 167.748562 P99 : 477.871174 P100 : 68906.000000 COUNT : 5983047 SUM : 269701721 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1011996 SUM : 1011996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3385649 SUM : 3493989768 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 224444.444444 P95 : 719833.333333 P99 : 974626.000000 P100 : 974626.000000 COUNT : 49 SUM : 13551209 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 271 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0