[2024-12-17 07:56:46.691309] Starting SPDK v24.09.1-pre git sha1 b18e1bd62 / DPDK 22.11.4 initialization... [2024-12-17 07:56:46.691450] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3278402 ] Set seed to 1734418606687842 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-17 07:56:46.773390] app.c: 917:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-17 07:56:46.811203] app.c: 610:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-17 07:56:46.811249] app.c: 611:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3278402' to capture a snapshot of events at runtime. [2024-12-17 07:56:46.811259] app.c: 616:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-12-17 07:56:46.811269] app.c: 617:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-12-17 07:56:46.811277] app.c: 618:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3278402 for offline analysis/debug. [2024-12-17 07:56:46.811360] reactor.c: 990:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Dec 17 07:56:51 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/17-07:56:58 ... thread 0: (1000000,1000000) ops and (127291.8,127291.8) ops/second in (7.855963,7.855963) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 15/11 3.04 GB 50.4 0.3 0.0 0.3 1.2 0.9 0.0 1.4 37.1 141.1 8.48 3.88 12 0.707 319K 359 0.0 0.0 L1 2/2 253.52 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/2 1.03 GB 1.0 2.0 1.0 1.0 2.0 1.0 0.0 2.0 94.2 93.3 21.93 10.75 12 1.827 981K 55 0.0 0.0 L4 166/6 9.90 GB 1.0 4.5 3.0 1.5 4.5 3.0 0.2 1.5 115.6 114.8 39.81 21.74 32 1.244 2173K 1839 0.0 0.0 L5 1124/0 92.34 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1328/21 106.55 GB 0.0 6.8 4.0 2.8 7.6 4.8 3.1 8.9 99.4 111.3 70.22 36.38 56 1.254 3474K 2253 0.0 0.0 Int 0/0 0.00 KB 0.0 6.8 4.0 2.8 7.6 4.8 3.1 8.9 99.4 111.3 70.22 36.38 56 1.254 3474K 2253 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.8 4.0 2.8 6.8 3.9 0.0 0.0 109.0 108.2 64.05 33.87 45 1.423 3474K 2253 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 142.9 6.17 2.51 11 0.561 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.0 total, 7.9 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.63 GB write, 972.07 MB/s write, 6.82 GB read, 868.61 MB/s read, 70.2 seconds Interval compaction: 7.63 GB write, 994.45 MB/s write, 6.82 GB read, 888.61 MB/s read, 70.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 19, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 324248 Average: 18.6632 StdDev: 231.01 Min: 1 Median: 1.9901 Max: 42006 Percentiles: P50: 1.99 P75: 2.98 P99: 352.18 P99.9: 2878.23 P99.99: 4975.16 ------------------------------------------------------ [ 0, 1 ] 10253 3.162% 3.162% # ( 1, 2 ] 153397 47.309% 50.471% ######### ( 2, 3 ] 80858 24.937% 75.408% ##### ( 3, 4 ] 30083 9.278% 84.685% ## ( 4, 6 ] 34158 10.535% 95.220% ## ( 6, 10 ] 9094 2.805% 98.025% # ( 10, 15 ] 1585 0.489% 98.513% ( 15, 22 ] 205 0.063% 98.577% ( 22, 34 ] 26 0.008% 98.585% ( 34, 51 ] 17 0.005% 98.590% ( 51, 76 ] 18 0.006% 98.596% ( 76, 110 ] 91 0.028% 98.624% ( 110, 170 ] 395 0.122% 98.745% ( 170, 250 ] 427 0.132% 98.877% ( 250, 380 ] 507 0.156% 99.033% ( 380, 580 ] 684 0.211% 99.244% ( 580, 870 ] 750 0.231% 99.476% ( 870, 1300 ] 657 0.203% 99.678% ( 1300, 1900 ] 393 0.121% 99.800% ( 1900, 2900 ] 333 0.103% 99.902% ( 2900, 4400 ] 260 0.080% 99.982% ( 4400, 6600 ] 94 0.029% 100.011% ( 6600, 9900 ] 14 0.004% 100.016% ( 9900, 14000 ] 10 0.003% 100.019% ( 14000, 22000 ] 6 0.002% 100.021% ( 22000, 33000 ] 2 0.001% 100.021% ( 33000, 50000 ] 1 0.000% 100.022% ** Level 1 read latency histogram (micros): Count: 44 Average: 1367.0455 StdDev: 1884.87 Min: 104 Median: 725.0000 Max: 10947 Percentiles: P50: 725.00 P75: 1540.00 P99: 10947.00 P99.9: 10947.00 P99.99: 10947.00 ------------------------------------------------------ ( 76, 110 ] 1 2.273% 2.273% ( 110, 170 ] 2 4.545% 6.818% # ( 170, 250 ] 5 11.364% 18.182% ## ( 250, 380 ] 4 9.091% 27.273% ## ( 380, 580 ] 7 15.909% 43.182% ### ( 580, 870 ] 6 13.636% 56.818% ### ( 870, 1300 ] 6 13.636% 70.455% ### ( 1300, 1900 ] 5 11.364% 81.818% ## ( 1900, 2900 ] 2 4.545% 86.364% # ( 2900, 4400 ] 3 6.818% 93.182% # ( 4400, 6600 ] 2 4.545% 97.727% # ( 9900, 14000 ] 1 2.273% 100.000% ** Level 2 read latency histogram (micros): Count: 203022 Average: 13.9445 StdDev: 235.76 Min: 0 Median: 2.0516 Max: 28929 Percentiles: P50: 2.05 P75: 3.15 P99: 12.28 P99.9: 1932.61 P99.99: 3613.58 ------------------------------------------------------ [ 0, 1 ] 5002 2.464% 2.464% ( 1, 2 ] 93928 46.265% 48.729% ######### ( 2, 3 ] 49987 24.621% 73.350% ##### ( 3, 4 ] 22491 11.078% 84.428% ## ( 4, 6 ] 22718 11.190% 95.618% ## ( 6, 10 ] 6313 3.110% 98.728% # ( 10, 15 ] 1214 0.598% 99.326% ( 15, 22 ] 186 0.092% 99.417% ( 22, 34 ] 39 0.019% 99.437% ( 34, 51 ] 16 0.008% 99.444% ( 51, 76 ] 12 0.006% 99.450% ( 76, 110 ] 10 0.005% 99.455% ( 110, 170 ] 19 0.009% 99.465% ( 170, 250 ] 51 0.025% 99.490% ( 250, 380 ] 107 0.053% 99.542% ( 380, 580 ] 209 0.103% 99.645% ( 580, 870 ] 201 0.099% 99.744% ( 870, 1300 ] 202 0.099% 99.844% ( 1300, 1900 ] 110 0.054% 99.898% ( 1900, 2900 ] 122 0.060% 99.958% ( 2900, 4400 ] 136 0.067% 100.025% ( 4400, 6600 ] 41 0.020% 100.045% ( 6600, 9900 ] 25 0.012% 100.058% ( 9900, 14000 ] 15 0.007% 100.065% ( 14000, 22000 ] 8 0.004% 100.069% ( 22000, 33000 ] 1 0.000% 100.069% ** Level 3 read latency histogram (micros): Count: 866395 Average: 12.5006 StdDev: 176.13 Min: 1 Median: 2.0214 Max: 26456 Percentiles: P50: 2.02 P75: 3.06 P99: 9.85 P99.9: 889.03 P99.99: 1245.73 ------------------------------------------------------ [ 0, 1 ] 13398 1.546% 1.546% ( 1, 2 ] 415190 47.922% 49.468% ########## ( 2, 3 ] 215181 24.836% 74.304% ##### ( 3, 4 ] 97934 11.304% 85.608% ## ( 4, 6 ] 95489 11.021% 96.629% ## ( 6, 10 ] 21354 2.465% 99.094% ( 10, 15 ] 4336 0.500% 99.595% ( 15, 22 ] 466 0.054% 99.648% ( 22, 34 ] 69 0.008% 99.656% ( 34, 51 ] 44 0.005% 99.661% ( 51, 76 ] 42 0.005% 99.666% ( 76, 110 ] 29 0.003% 99.670% ( 110, 170 ] 83 0.010% 99.679% ( 170, 250 ] 134 0.015% 99.695% ( 250, 380 ] 306 0.035% 99.730% ( 380, 580 ] 543 0.063% 99.793% ( 580, 870 ] 889 0.103% 99.895% ( 870, 1300 ] 940 0.108% 100.004% ( 1300, 1900 ] 714 0.082% 100.086% ( 1900, 2900 ] 621 0.072% 100.158% ( 2900, 4400 ] 590 0.068% 100.226% ( 4400, 6600 ] 190 0.022% 100.248% ( 6600, 9900 ] 40 0.005% 100.252% ( 9900, 14000 ] 6 0.001% 100.253% ( 14000, 22000 ] 12 0.001% 100.255% ( 22000, 33000 ] 2 0.000% 100.255% ** Level 4 read latency histogram (micros): Count: 416133 Average: 22.3347 StdDev: 257.79 Min: 1 Median: 1.9552 Max: 45359 Percentiles: P50: 1.96 P75: 2.94 P99: 473.53 P99.9: 2487.65 P99.99: 3433.14 ------------------------------------------------------ [ 0, 1 ] 7379 1.773% 1.773% ( 1, 2 ] 210097 50.488% 52.261% ########## ( 2, 3 ] 100840 24.233% 76.494% ##### ( 3, 4 ] 42860 10.300% 86.793% ## ( 4, 6 ] 37925 9.114% 95.907% ## ( 6, 10 ] 9469 2.275% 98.183% ( 10, 15 ] 1764 0.424% 98.606% ( 15, 22 ] 174 0.042% 98.648% ( 22, 34 ] 26 0.006% 98.655% ( 34, 51 ] 24 0.006% 98.660% ( 51, 76 ] 31 0.007% 98.668% ( 76, 110 ] 47 0.011% 98.679% ( 110, 170 ] 241 0.058% 98.737% ( 170, 250 ] 347 0.083% 98.820% ( 250, 380 ] 403 0.097% 98.917% ( 380, 580 ] 737 0.177% 99.094% ( 580, 870 ] 1071 0.257% 99.352% ( 870, 1300 ] 1132 0.272% 99.624% ( 1300, 1900 ] 839 0.202% 99.825% ( 1900, 2900 ] 529 0.127% 99.952% ( 2900, 4400 ] 440 0.106% 100.058% ( 4400, 6600 ] 169 0.041% 100.099% ( 6600, 9900 ] 30 0.007% 100.106% ( 9900, 14000 ] 1 0.000% 100.106% ( 14000, 22000 ] 6 0.001% 100.108% ( 22000, 33000 ] 3 0.001% 100.108% ( 33000, 50000 ] 3 0.001% 100.109% ** Level 5 read latency histogram (micros): Count: 16188 Average: 1386.5500 StdDev: 1825.58 Min: 79 Median: 808.3581 Max: 65644 Percentiles: P50: 808.36 P75: 1766.96 P99: 6560.97 P99.9: 20249.60 P99.99: 54765.00 ------------------------------------------------------ ( 76, 110 ] 253 1.563% 1.563% ( 110, 170 ] 1220 7.536% 9.099% ## ( 170, 250 ] 1292 7.981% 17.081% ## ( 250, 380 ] 1451 8.963% 26.044% ## ( 380, 580 ] 2022 12.491% 38.535% ## ( 580, 870 ] 2357 14.560% 53.095% ### ( 870, 1300 ] 2307 14.251% 67.346% ### ( 1300, 1900 ] 1592 9.834% 77.181% ## ( 1900, 2900 ] 1366 8.438% 85.619% ## ( 2900, 4400 ] 1453 8.976% 94.595% ## ( 4400, 6600 ] 726 4.485% 99.080% # ( 6600, 9900 ] 107 0.661% 99.741% ( 9900, 14000 ] 18 0.111% 99.852% ( 14000, 22000 ] 10 0.062% 99.914% ( 22000, 33000 ] 10 0.062% 99.975% ( 33000, 50000 ] 2 0.012% 99.988% ( 50000, 75000 ] 2 0.012% 100.000% ** DB Stats ** Uptime(secs): 8.0 total, 7.9 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 122.43 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.850 H:M:S, 47.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 125.24 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.850 H:M:S, 49.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/17-07:57:02 ... thread 0: (1000000,2000000) ops and (290579.8,177032.5) ops/second in (3.441396,11.297359) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/11 4.02 GB 163.6 1.4 0.0 1.4 3.3 1.8 0.0 1.8 88.9 204.8 16.29 11.05 32 0.509 1471K 1706 0.0 0.0 L1 2/2 253.52 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 967.70 MB 1.0 2.0 1.0 1.0 2.0 1.0 0.0 2.0 94.2 93.3 21.93 10.75 12 1.827 981K 55 0.0 0.0 L4 163/0 9.67 GB 1.0 4.9 3.1 1.8 4.8 3.0 0.2 1.6 119.2 118.4 41.91 23.51 33 1.270 2173K 1839 0.0 0.0 L5 1128/0 92.64 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1331/13 107.53 GB 0.0 8.3 4.1 4.2 10.1 5.9 3.4 5.5 106.2 129.1 80.13 45.31 77 1.041 4626K 3600 0.0 0.0 Int 0/0 0.00 KB 0.0 8.3 4.1 4.2 10.1 5.9 3.4 5.5 106.2 129.1 80.13 45.31 77 1.041 4626K 3600 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.3 4.1 4.2 8.3 4.0 0.0 0.0 120.3 119.5 70.76 40.02 50 1.415 4626K 3600 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 201.7 9.37 5.29 27 0.347 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.5 total, 11.3 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.10 GB write, 901.03 MB/s write, 8.31 GB read, 741.36 MB/s read, 80.1 seconds Interval compaction: 10.10 GB write, 915.45 MB/s write, 8.31 GB read, 753.23 MB/s read, 80.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 11, total-stops: 19, interval: 30 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 663473 Average: 15.9895 StdDev: 168.25 Min: 0 Median: 1.7433 Max: 42006 Percentiles: P50: 1.74 P75: 2.64 P99: 296.76 P99.9: 1443.77 P99.99: 2717.20 ------------------------------------------------------ [ 0, 1 ] 73613 11.095% 11.095% ## ( 1, 2 ] 347280 52.343% 63.438% ########## ( 2, 3 ] 119691 18.040% 81.478% #### ( 3, 4 ] 44932 6.772% 88.250% # ( 4, 6 ] 43141 6.502% 94.752% # ( 6, 10 ] 11319 1.706% 96.458% ( 10, 15 ] 1969 0.297% 96.755% ( 15, 22 ] 346 0.052% 96.807% ( 22, 34 ] 44 0.007% 96.814% ( 34, 51 ] 23 0.003% 96.818% ( 51, 76 ] 20 0.003% 96.821% ( 76, 110 ] 1950 0.294% 97.114% ( 110, 170 ] 7359 1.109% 98.224% ( 170, 250 ] 4279 0.645% 98.869% ( 250, 380 ] 2428 0.366% 99.234% ( 380, 580 ] 2164 0.326% 99.561% ( 580, 870 ] 1361 0.205% 99.766% ( 870, 1300 ] 794 0.120% 99.885% ( 1300, 1900 ] 407 0.061% 99.947% ( 1900, 2900 ] 352 0.053% 100.000% ( 2900, 4400 ] 265 0.040% 100.040% ( 4400, 6600 ] 94 0.014% 100.054% ( 6600, 9900 ] 15 0.002% 100.056% ( 9900, 14000 ] 10 0.002% 100.058% ( 14000, 22000 ] 6 0.001% 100.059% ( 22000, 33000 ] 2 0.000% 100.059% ( 33000, 50000 ] 1 0.000% 100.059% ** Level 1 read latency histogram (micros): Count: 150 Average: 555.3400 StdDev: 1155.73 Min: 94 Median: 197.8261 Max: 10947 Percentiles: P50: 197.83 P75: 480.00 P99: 6050.00 P99.9: 10947.00 P99.99: 10947.00 ------------------------------------------------------ ( 76, 110 ] 13 8.667% 8.667% ## ( 110, 170 ] 54 36.000% 44.667% ####### ( 170, 250 ] 23 15.333% 60.000% ### ( 250, 380 ] 15 10.000% 70.000% ## ( 380, 580 ] 15 10.000% 80.000% ## ( 580, 870 ] 10 6.667% 86.667% # ( 870, 1300 ] 7 4.667% 91.333% # ( 1300, 1900 ] 5 3.333% 94.667% # ( 1900, 2900 ] 2 1.333% 96.000% ( 2900, 4400 ] 3 2.000% 98.000% ( 4400, 6600 ] 2 1.333% 99.333% ( 9900, 14000 ] 1 0.667% 100.000% ** Level 2 read latency histogram (micros): Count: 203022 Average: 13.9445 StdDev: 235.76 Min: 0 Median: 2.0516 Max: 28929 Percentiles: P50: 2.05 P75: 3.15 P99: 12.28 P99.9: 1932.61 P99.99: 3613.58 ------------------------------------------------------ [ 0, 1 ] 5002 2.464% 2.464% ( 1, 2 ] 93928 46.265% 48.729% ######### ( 2, 3 ] 49987 24.621% 73.350% ##### ( 3, 4 ] 22491 11.078% 84.428% ## ( 4, 6 ] 22718 11.190% 95.618% ## ( 6, 10 ] 6313 3.110% 98.728% # ( 10, 15 ] 1214 0.598% 99.326% ( 15, 22 ] 186 0.092% 99.417% ( 22, 34 ] 39 0.019% 99.437% ( 34, 51 ] 16 0.008% 99.444% ( 51, 76 ] 12 0.006% 99.450% ( 76, 110 ] 10 0.005% 99.455% ( 110, 170 ] 19 0.009% 99.465% ( 170, 250 ] 51 0.025% 99.490% ( 250, 380 ] 107 0.053% 99.542% ( 380, 580 ] 209 0.103% 99.645% ( 580, 870 ] 201 0.099% 99.744% ( 870, 1300 ] 202 0.099% 99.844% ( 1300, 1900 ] 110 0.054% 99.898% ( 1900, 2900 ] 122 0.060% 99.958% ( 2900, 4400 ] 136 0.067% 100.025% ( 4400, 6600 ] 41 0.020% 100.045% ( 6600, 9900 ] 25 0.012% 100.058% ( 9900, 14000 ] 15 0.007% 100.065% ( 14000, 22000 ] 8 0.004% 100.069% ( 22000, 33000 ] 1 0.000% 100.069% ** Level 3 read latency histogram (micros): Count: 881406 Average: 12.5007 StdDev: 174.79 Min: 1 Median: 2.0104 Max: 26456 Percentiles: P50: 2.01 P75: 3.05 P99: 9.95 P99.9: 885.69 P99.99: 1245.51 ------------------------------------------------------ [ 0, 1 ] 14215 1.613% 1.613% ( 1, 2 ] 424211 48.129% 49.742% ########## ( 2, 3 ] 217962 24.729% 74.471% ##### ( 3, 4 ] 98730 11.201% 85.672% ## ( 4, 6 ] 96118 10.905% 96.577% ## ( 6, 10 ] 21621 2.453% 99.030% ( 10, 15 ] 4356 0.494% 99.524% ( 15, 22 ] 472 0.054% 99.578% ( 22, 34 ] 69 0.008% 99.586% ( 34, 51 ] 44 0.005% 99.591% ( 51, 76 ] 42 0.005% 99.595% ( 76, 110 ] 82 0.009% 99.605% ( 110, 170 ] 388 0.044% 99.649% ( 170, 250 ] 288 0.033% 99.681% ( 250, 380 ] 381 0.043% 99.725% ( 380, 580 ] 599 0.068% 99.793% ( 580, 870 ] 912 0.103% 99.896% ( 870, 1300 ] 948 0.108% 100.004% ( 1300, 1900 ] 714 0.081% 100.085% ( 1900, 2900 ] 621 0.070% 100.155% ( 2900, 4400 ] 590 0.067% 100.222% ( 4400, 6600 ] 190 0.022% 100.244% ( 6600, 9900 ] 40 0.005% 100.248% ( 9900, 14000 ] 6 0.001% 100.249% ( 14000, 22000 ] 12 0.001% 100.250% ( 22000, 33000 ] 2 0.000% 100.250% ** Level 4 read latency histogram (micros): Count: 474730 Average: 22.4771 StdDev: 243.65 Min: 1 Median: 1.9356 Max: 45359 Percentiles: P50: 1.94 P75: 2.92 P99: 459.49 P99.9: 2375.08 P99.99: 3405.19 ------------------------------------------------------ [ 0, 1 ] 9628 2.028% 2.028% ( 1, 2 ] 243406 51.273% 53.301% ########## ( 2, 3 ] 111900 23.571% 76.872% ##### ( 3, 4 ] 45892 9.667% 86.539% ## ( 4, 6 ] 40267 8.482% 95.021% ## ( 6, 10 ] 10499 2.212% 97.233% ( 10, 15 ] 1864 0.393% 97.625% ( 15, 22 ] 196 0.041% 97.666% ( 22, 34 ] 27 0.006% 97.672% ( 34, 51 ] 25 0.005% 97.677% ( 51, 76 ] 32 0.007% 97.684% ( 76, 110 ] 637 0.134% 97.818% ( 110, 170 ] 2586 0.545% 98.363% ( 170, 250 ] 1575 0.332% 98.695% ( 250, 380 ] 961 0.202% 98.897% ( 380, 580 ] 1227 0.258% 99.156% ( 580, 870 ] 1257 0.265% 99.421% ( 870, 1300 ] 1175 0.248% 99.668% ( 1300, 1900 ] 849 0.179% 99.847% ( 1900, 2900 ] 531 0.112% 99.959% ( 2900, 4400 ] 441 0.093% 100.052% ( 4400, 6600 ] 169 0.036% 100.087% ( 6600, 9900 ] 30 0.006% 100.094% ( 9900, 14000 ] 2 0.000% 100.094% ( 14000, 22000 ] 6 0.001% 100.095% ( 22000, 33000 ] 3 0.001% 100.096% ( 33000, 50000 ] 3 0.001% 100.096% ** Level 5 read latency histogram (micros): Count: 54344 Average: 575.1346 StdDev: 1141.47 Min: 75 Median: 216.3175 Max: 65644 Percentiles: P50: 216.32 P75: 512.09 P99: 5413.82 P99.9: 9556.56 P99.99: 30322.16 ------------------------------------------------------ ( 51, 76 ] 2 0.004% 0.004% ( 76, 110 ] 4394 8.086% 8.089% ## ( 110, 170 ] 17050 31.374% 39.463% ###### ( 170, 250 ] 9890 18.199% 57.662% #### ( 250, 380 ] 5796 10.665% 68.328% ## ( 380, 580 ] 5490 10.102% 78.430% ## ( 580, 870 ] 3736 6.875% 85.305% # ( 870, 1300 ] 2597 4.779% 90.084% # ( 1300, 1900 ] 1644 3.025% 93.109% # ( 1900, 2900 ] 1404 2.584% 95.692% # ( 2900, 4400 ] 1463 2.692% 98.384% # ( 4400, 6600 ] 726 1.336% 99.720% ( 6600, 9900 ] 109 0.201% 99.921% ( 9900, 14000 ] 20 0.037% 99.958% ( 14000, 22000 ] 10 0.018% 99.976% ( 22000, 33000 ] 10 0.018% 99.994% ( 33000, 50000 ] 2 0.004% 99.998% ( 50000, 75000 ] 2 0.004% 100.002% ** DB Stats ** Uptime(secs): 11.5 total, 3.4 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 171.46 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.850 H:M:S, 33.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 285.99 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/17-07:57:08 ... thread 0: (1000000,3000000) ops and (167271.1,173654.5) ops/second in (5.978320,17.275679) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/14 4.94 GB 125.8 2.6 0.0 2.6 5.5 2.8 0.0 1.9 111.4 230.4 24.32 18.55 53 0.459 2750K 2922 0.0 0.0 L1 2/2 253.52 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 19/0 967.70 MB 1.0 2.0 1.0 1.0 2.0 1.0 0.0 2.0 94.2 93.3 21.93 10.75 12 1.827 981K 55 0.0 0.0 L4 163/0 9.67 GB 1.0 4.9 3.1 1.8 4.8 3.0 0.2 1.6 119.2 118.4 41.91 23.51 33 1.270 2173K 1839 0.0 0.0 L5 1128/0 92.64 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.3 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1331/16 108.45 GB 0.0 9.5 4.1 5.5 12.3 6.9 3.4 4.4 110.8 143.0 88.15 52.81 98 0.900 5905K 4816 0.0 0.0 Int 0/0 0.00 KB 0.0 9.5 4.1 5.5 12.3 6.9 3.4 4.4 110.8 143.0 88.15 52.81 98 0.900 5905K 4816 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 9.5 4.1 5.5 9.5 4.0 0.0 0.0 129.1 128.3 75.68 44.84 55 1.376 5905K 4816 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 232.3 12.47 7.97 43 0.290 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 17.5 total, 17.3 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.31 GB write, 722.32 MB/s write, 9.54 GB read, 559.66 MB/s read, 88.2 seconds Interval compaction: 12.31 GB write, 729.88 MB/s write, 9.54 GB read, 565.52 MB/s read, 88.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 10, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 10, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 19, pending-compaction-bytes-delays: 11, pending-compaction-bytes-stops: 0, total-delays: 21, total-stops: 19, interval: 40 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1048028 Average: 17.3074 StdDev: 139.89 Min: 0 Median: 1.6427 Max: 42006 Percentiles: P50: 1.64 P75: 2.46 P99: 285.24 P99.9: 1045.15 P99.99: 1924.40 ------------------------------------------------------ [ 0, 1 ] 175177 16.715% 16.715% ### ( 1, 2 ] 542747 51.787% 68.502% ########## ( 2, 3 ] 147259 14.051% 82.553% ### ( 3, 4 ] 57646 5.500% 88.054% # ( 4, 6 ] 49818 4.753% 92.807% # ( 6, 10 ] 12804 1.222% 94.029% ( 10, 15 ] 2510 0.239% 94.269% ( 15, 22 ] 463 0.044% 94.313% ( 22, 34 ] 71 0.007% 94.320% ( 34, 51 ] 29 0.003% 94.322% ( 51, 76 ] 30 0.003% 94.325% ( 76, 110 ] 8481 0.809% 95.134% ( 110, 170 ] 27936 2.666% 97.800% # ( 170, 250 ] 11307 1.079% 98.879% ( 250, 380 ] 4695 0.448% 99.327% ( 380, 580 ] 3827 0.365% 99.692% ( 580, 870 ] 1829 0.175% 99.867% ( 870, 1300 ] 869 0.083% 99.949% ( 1300, 1900 ] 419 0.040% 99.989% ( 1900, 2900 ] 377 0.036% 100.025% ( 2900, 4400 ] 272 0.026% 100.051% ( 4400, 6600 ] 94 0.009% 100.060% ( 6600, 9900 ] 15 0.001% 100.062% ( 9900, 14000 ] 10 0.001% 100.063% ( 14000, 22000 ] 6 0.001% 100.063% ( 22000, 33000 ] 2 0.000% 100.063% ( 33000, 50000 ] 1 0.000% 100.064% ** Level 1 read latency histogram (micros): Count: 457 Average: 296.7112 StdDev: 690.23 Min: 86 Median: 154.7982 Max: 10947 Percentiles: P50: 154.80 P75: 227.04 P99: 3615.00 P99.9: 10947.00 P99.99: 10947.00 ------------------------------------------------------ ( 76, 110 ] 62 13.567% 13.567% ### ( 110, 170 ] 223 48.796% 62.363% ########## ( 170, 250 ] 81 17.724% 80.088% #### ( 250, 380 ] 30 6.565% 86.652% # ( 380, 580 ] 30 6.565% 93.217% # ( 580, 870 ] 11 2.407% 95.624% ( 870, 1300 ] 7 1.532% 97.155% ( 1300, 1900 ] 5 1.094% 98.249% ( 1900, 2900 ] 2 0.438% 98.687% ( 2900, 4400 ] 3 0.656% 99.344% ( 4400, 6600 ] 2 0.438% 99.781% ( 9900, 14000 ] 1 0.219% 100.000% ** Level 2 read latency histogram (micros): Count: 203022 Average: 13.9445 StdDev: 235.76 Min: 0 Median: 2.0516 Max: 28929 Percentiles: P50: 2.05 P75: 3.15 P99: 12.28 P99.9: 1932.61 P99.99: 3613.58 ------------------------------------------------------ [ 0, 1 ] 5002 2.464% 2.464% ( 1, 2 ] 93928 46.265% 48.729% ######### ( 2, 3 ] 49987 24.621% 73.350% ##### ( 3, 4 ] 22491 11.078% 84.428% ## ( 4, 6 ] 22718 11.190% 95.618% ## ( 6, 10 ] 6313 3.110% 98.728% # ( 10, 15 ] 1214 0.598% 99.326% ( 15, 22 ] 186 0.092% 99.417% ( 22, 34 ] 39 0.019% 99.437% ( 34, 51 ] 16 0.008% 99.444% ( 51, 76 ] 12 0.006% 99.450% ( 76, 110 ] 10 0.005% 99.455% ( 110, 170 ] 19 0.009% 99.465% ( 170, 250 ] 51 0.025% 99.490% ( 250, 380 ] 107 0.053% 99.542% ( 380, 580 ] 209 0.103% 99.645% ( 580, 870 ] 201 0.099% 99.744% ( 870, 1300 ] 202 0.099% 99.844% ( 1300, 1900 ] 110 0.054% 99.898% ( 1900, 2900 ] 122 0.060% 99.958% ( 2900, 4400 ] 136 0.067% 100.025% ( 4400, 6600 ] 41 0.020% 100.045% ( 6600, 9900 ] 25 0.012% 100.058% ( 9900, 14000 ] 15 0.007% 100.065% ( 14000, 22000 ] 8 0.004% 100.069% ( 22000, 33000 ] 1 0.000% 100.069% ** Level 3 read latency histogram (micros): Count: 883052 Average: 12.7913 StdDev: 174.82 Min: 1 Median: 2.0142 Max: 26456 Percentiles: P50: 2.01 P75: 3.06 P99: 11.51 P99.9: 886.71 P99.99: 1246.06 ------------------------------------------------------ [ 0, 1 ] 14215 1.610% 1.610% ( 1, 2 ] 424219 48.040% 49.650% ########## ( 2, 3 ] 217980 24.685% 74.335% ##### ( 3, 4 ] 98734 11.181% 85.516% ## ( 4, 6 ] 96137 10.887% 96.403% ## ( 6, 10 ] 21622 2.449% 98.851% ( 10, 15 ] 4356 0.493% 99.344% ( 15, 22 ] 472 0.053% 99.398% ( 22, 34 ] 69 0.008% 99.406% ( 34, 51 ] 44 0.005% 99.411% ( 51, 76 ] 42 0.005% 99.415% ( 76, 110 ] 380 0.043% 99.458% ( 110, 170 ] 1243 0.141% 99.599% ( 170, 250 ] 550 0.062% 99.662% ( 250, 380 ] 473 0.054% 99.715% ( 380, 580 ] 665 0.075% 99.790% ( 580, 870 ] 931 0.105% 99.896% ( 870, 1300 ] 951 0.108% 100.004% ( 1300, 1900 ] 715 0.081% 100.084% ( 1900, 2900 ] 621 0.070% 100.155% ( 2900, 4400 ] 590 0.067% 100.222% ( 4400, 6600 ] 190 0.022% 100.243% ( 6600, 9900 ] 40 0.005% 100.248% ( 9900, 14000 ] 6 0.001% 100.248% ( 14000, 22000 ] 12 0.001% 100.250% ( 22000, 33000 ] 2 0.000% 100.250% ** Level 4 read latency histogram (micros): Count: 488437 Average: 26.3243 StdDev: 242.11 Min: 1 Median: 1.9632 Max: 45359 Percentiles: P50: 1.96 P75: 3.02 P99: 494.41 P99.9: 2359.30 P99.99: 3402.79 ------------------------------------------------------ [ 0, 1 ] 9629 1.971% 1.971% ( 1, 2 ] 243541 49.861% 51.833% ########## ( 2, 3 ] 112222 22.976% 74.808% ##### ( 3, 4 ] 46072 9.433% 84.241% ## ( 4, 6 ] 40498 8.291% 92.532% ## ( 6, 10 ] 10528 2.155% 94.688% ( 10, 15 ] 1873 0.383% 95.071% ( 15, 22 ] 196 0.040% 95.111% ( 22, 34 ] 27 0.006% 95.117% ( 34, 51 ] 25 0.005% 95.122% ( 51, 76 ] 33 0.007% 95.129% ( 76, 110 ] 3073 0.629% 95.758% ( 110, 170 ] 9522 1.949% 97.707% ( 170, 250 ] 3699 0.757% 98.465% ( 250, 380 ] 1621 0.332% 98.797% ( 380, 580 ] 1737 0.356% 99.152% ( 580, 870 ] 1357 0.278% 99.430% ( 870, 1300 ] 1194 0.244% 99.674% ( 1300, 1900 ] 854 0.175% 99.849% ( 1900, 2900 ] 539 0.110% 99.960% ( 2900, 4400 ] 442 0.090% 100.050% ( 4400, 6600 ] 169 0.035% 100.085% ( 6600, 9900 ] 30 0.006% 100.091% ( 9900, 14000 ] 2 0.000% 100.091% ( 14000, 22000 ] 6 0.001% 100.093% ( 22000, 33000 ] 3 0.001% 100.093% ( 33000, 50000 ] 3 0.001% 100.094% ** Level 5 read latency histogram (micros): Count: 141863 Average: 326.1607 StdDev: 738.64 Min: 71 Median: 156.6870 Max: 65644 Percentiles: P50: 156.69 P75: 239.45 P99: 3843.88 P99.9: 6785.80 P99.99: 17850.96 ------------------------------------------------------ ( 51, 76 ] 13 0.009% 0.009% ( 76, 110 ] 20281 14.296% 14.305% ### ( 110, 170 ] 65077 45.873% 60.178% ######### ( 170, 250 ] 24221 17.074% 77.252% ### ( 250, 380 ] 10373 7.312% 84.564% # ( 380, 580 ] 9099 6.414% 90.978% # ( 580, 870 ] 4635 3.267% 94.245% # ( 870, 1300 ] 2720 1.917% 96.162% ( 1300, 1900 ] 1673 1.179% 97.342% ( 1900, 2900 ] 1428 1.007% 98.348% ( 2900, 4400 ] 1469 1.036% 99.384% ( 4400, 6600 ] 726 0.512% 99.896% ( 6600, 9900 ] 109 0.077% 99.973% ( 9900, 14000 ] 20 0.014% 99.987% ( 14000, 22000 ] 10 0.007% 99.994% ( 22000, 33000 ] 10 0.007% 100.001% ( 33000, 50000 ] 2 0.001% 100.002% ( 50000, 75000 ] 2 0.001% 100.004% ** DB Stats ** Uptime(secs): 17.5 total, 6.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 169.12 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.306 H:M:S, 36.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 164.63 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.456 H:M:S, 41.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 221.169 micros/op 18071 ops/sec 60.150 seconds 1086996 operations; 17.5 MB/s (273999 of 273999 found) Microseconds per read: Count: 1086996 Average: 221.1702 StdDev: 713.97 Min: 2 Median: 167.1035 Max: 214602 Percentiles: P50: 167.10 P75: 226.17 P99: 819.46 P99.9: 10610.04 P99.99: 27632.92 ------------------------------------------------------ ( 1, 2 ] 7 0.001% 0.001% ( 2, 3 ] 43 0.004% 0.005% ( 3, 4 ] 57 0.005% 0.010% ( 4, 6 ] 87 0.008% 0.018% ( 6, 10 ] 389 0.036% 0.054% ( 10, 15 ] 3522 0.324% 0.378% ( 15, 22 ] 4994 0.459% 0.837% ( 22, 34 ] 10364 0.953% 1.791% ( 34, 51 ] 2290 0.211% 2.001% ( 51, 76 ] 283 0.026% 2.027% ( 76, 110 ] 27583 2.538% 4.565% # ( 110, 170 ] 518930 47.740% 52.305% ########## ( 170, 250 ] 351366 32.324% 84.629% ###### ( 250, 380 ] 107432 9.883% 94.512% ## ( 380, 580 ] 38266 3.520% 98.033% # ( 580, 870 ] 12732 1.171% 99.204% ( 870, 1300 ] 4018 0.370% 99.574% ( 1300, 1900 ] 1434 0.132% 99.706% ( 1900, 2900 ] 941 0.087% 99.792% ( 2900, 4400 ] 690 0.063% 99.856% ( 4400, 6600 ] 258 0.024% 99.879% ( 6600, 9900 ] 152 0.014% 99.893% ( 9900, 14000 ] 410 0.038% 99.931% ( 14000, 22000 ] 563 0.052% 99.983% ( 22000, 33000 ] 149 0.014% 99.997% ( 33000, 50000 ] 28 0.003% 99.999% ( 50000, 75000 ] 3 0.000% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 110000, 170000 ] 2 0.000% 100.000% ( 170000, 250000 ] 2 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 17155772 rocksdb.bloom.filter.full.positive COUNT : 1253220 rocksdb.bloom.filter.full.true.positive COUNT : 1086559 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 437 rocksdb.memtable.miss COUNT : 1086559 rocksdb.l0.hit COUNT : 47925 rocksdb.l1.hit COUNT : 13433 rocksdb.l2andup.hit COUNT : 1025201 rocksdb.compaction.key.drop.new COUNT : 107397 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3652583 rocksdb.number.keys.read COUNT : 1086996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3769465656 rocksdb.bytes.read COUNT : 1086996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1535 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 45495924 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 15878498994 rocksdb.compact.write.bytes COUNT : 15764749825 rocksdb.flush.write.bytes COUNT : 3765409098 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 23 rocksdb.number.superversion_acquires COUNT : 727 rocksdb.number.superversion_releases COUNT : 688 rocksdb.number.superversion_cleanups COUNT : 181 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 187 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3734914048 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1337344 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 20771354652 rocksdb.non.last.level.read.count COUNT : 4874230 rocksdb.block.checksum.compute.count COUNT : 5109492 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 166.235175 P95 : 405.894498 P99 : 818.691110 P100 : 214600.000000 COUNT : 1086996 SUM : 239483134 rocksdb.db.write.micros P50 : 1.560080 P95 : 5.336956 P99 : 18.415129 P100 : 787147.000000 COUNT : 3652583 SUM : 55197437 rocksdb.compaction.times.micros P50 : 1270000.000000 P95 : 2832142.857143 P99 : 38732839.000000 P100 : 38732839.000000 COUNT : 59 SUM : 122124415 rocksdb.compaction.times.cpu_micros P50 : 867083.333333 P95 : 1733750.000000 P99 : 14385646.000000 P100 : 14385646.000000 COUNT : 59 SUM : 66837760 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 764.545455 P95 : 119000.000000 P99 : 260452.000000 P100 : 260452.000000 COUNT : 54 SUM : 612000 rocksdb.compaction.outfile.sync.micros P50 : 16857.142857 P95 : 204666.666667 P99 : 363966.666667 P100 : 449020.000000 COUNT : 174 SUM : 8400164 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 184.285714 P95 : 2887.500000 P99 : 6347.000000 P100 : 8810.000000 COUNT : 123 SUM : 99255 rocksdb.table.open.io.micros P50 : 9577.951807 P95 : 20472.000000 P99 : 28256.250000 P100 : 132875.000000 COUNT : 1535 SUM : 15244235 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.968093 P95 : 115.937686 P99 : 222.333895 P100 : 42012.000000 COUNT : 3844751 SUM : 77228000 rocksdb.read.block.get.micros P50 : 137.951297 P95 : 287.390647 P99 : 744.852454 P100 : 65707.000000 COUNT : 1263214 SUM : 202856297 rocksdb.write.raw.block.micros P50 : 0.533495 P95 : 1.296872 P99 : 3.625412 P100 : 53159.000000 COUNT : 4730339 SUM : 13032667 rocksdb.numfiles.in.singlecompaction P50 : 2.833333 P95 : 6.600000 P99 : 36.000000 P100 : 36.000000 COUNT : 71 SUM : 267 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.569379 P95 : 1249.939668 P99 : 2660.194117 P100 : 786956.000000 COUNT : 252532 SUM : 45520732 rocksdb.sst.read.micros P50 : 2.409445 P95 : 176.620075 P99 : 457.177186 P100 : 65644.000000 COUNT : 4874279 SUM : 265321825 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1086996 SUM : 1086996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3652583 SUM : 3769465656 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 218000.000000 P95 : 894000.000000 P99 : 1027451.000000 P100 : 1027451.000000 COUNT : 54 SUM : 14739817 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 189 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0