[2024-12-15 13:43:44.546070] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 22.11.4 initialization... [2024-12-15 13:43:44.546222] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid712361 ] Set seed to 1734266624542589 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-15 13:43:44.631088] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-15 13:43:44.670902] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-15 13:43:44.670951] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 712361' to capture a snapshot of events at runtime. [2024-12-15 13:43:44.670964] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid712361 for offline analysis/debug. [2024-12-15 13:43:44.671008] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Sun Dec 15 13:43:48 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/15-13:43:56 ... thread 0: (1000000,1000000) ops and (130945.6,130945.6) ops/second in (7.636758,7.636758) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 13/11 1.82 GB 50.3 0.4 0.0 0.4 1.2 0.8 0.0 1.5 40.6 128.6 9.30 3.93 10 0.930 383K 502 0.0 0.0 L1 7/7 830.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 83.61 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 28/8 1.76 GB 1.3 2.4 1.5 1.0 2.4 1.4 0.0 1.7 104.5 103.5 23.80 11.25 10 2.380 820K 0 0.0 0.0 L4 195/19 11.84 GB 1.1 4.0 2.5 1.6 4.0 2.4 0.0 1.6 110.1 108.7 37.27 19.02 31 1.202 1905K 0 0.0 0.0 L5 1104/0 90.68 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1348/45 106.98 GB 0.0 6.8 3.9 2.9 7.5 4.6 0.0 9.4 99.1 109.6 70.37 34.21 51 1.380 3109K 502 0.0 0.0 Int 0/0 0.00 KB 0.0 6.8 3.9 2.9 7.5 4.6 0.0 9.4 99.1 109.6 70.37 34.21 51 1.380 3109K 502 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.8 3.9 2.9 6.7 3.8 0.0 0.0 109.7 108.5 63.53 31.80 42 1.513 3109K 502 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 119.8 6.84 2.41 9 0.760 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.8 total, 7.6 interval Flush(GB): cumulative 0.800, interval 0.800 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.53 GB write, 985.99 MB/s write, 6.81 GB read, 891.35 MB/s read, 70.4 seconds Interval compaction: 7.53 GB write, 1009.49 MB/s write, 6.81 GB read, 912.59 MB/s read, 70.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 28, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 28, interval: 28 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 309314 Average: 14.2374 StdDev: 178.56 Min: 1 Median: 1.8531 Max: 45624 Percentiles: P50: 1.85 P75: 2.92 P99: 12.72 P99.9: 2332.57 P99.99: 3818.89 ------------------------------------------------------ [ 0, 1 ] 18958 6.129% 6.129% # ( 1, 2 ] 159063 51.424% 57.553% ########## ( 2, 3 ] 58579 18.938% 76.492% #### ( 3, 4 ] 29960 9.686% 86.178% ## ( 4, 6 ] 32119 10.384% 96.562% ## ( 6, 10 ] 6907 2.233% 98.795% ( 10, 15 ] 1167 0.377% 99.172% ( 15, 22 ] 110 0.036% 99.208% ( 22, 34 ] 12 0.004% 99.211% ( 34, 51 ] 7 0.002% 99.214% ( 51, 76 ] 15 0.005% 99.219% ( 76, 110 ] 6 0.002% 99.221% ( 110, 170 ] 19 0.006% 99.227% ( 170, 250 ] 62 0.020% 99.247% ( 250, 380 ] 130 0.042% 99.289% ( 380, 580 ] 296 0.096% 99.384% ( 580, 870 ] 492 0.159% 99.544% ( 870, 1300 ] 533 0.172% 99.716% ( 1300, 1900 ] 449 0.145% 99.861% ( 1900, 2900 ] 279 0.090% 99.951% ( 2900, 4400 ] 196 0.063% 100.015% ( 4400, 6600 ] 52 0.017% 100.031% ( 6600, 9900 ] 7 0.002% 100.034% ( 33000, 50000 ] 1 0.000% 100.034% ** Level 1 read latency histogram (micros): Count: 57950 Average: 15.4073 StdDev: 189.58 Min: 1 Median: 1.9866 Max: 10779 Percentiles: P50: 1.99 P75: 3.27 P99: 14.10 P99.9: 3444.40 P99.99: 6490.69 ------------------------------------------------------ [ 0, 1 ] 1911 3.298% 3.298% # ( 1, 2 ] 27432 47.337% 50.635% ######### ( 2, 3 ] 12409 21.413% 72.048% #### ( 3, 4 ] 6243 10.773% 82.821% ## ( 4, 6 ] 7122 12.290% 95.111% ## ( 6, 10 ] 1927 3.325% 98.437% # ( 10, 15 ] 398 0.687% 99.123% ( 15, 22 ] 63 0.109% 99.232% ( 22, 34 ] 18 0.031% 99.263% ( 34, 51 ] 2 0.003% 99.267% ( 51, 76 ] 3 0.005% 99.272% ( 76, 110 ] 2 0.003% 99.275% ( 110, 170 ] 3 0.005% 99.280% ( 170, 250 ] 9 0.016% 99.296% ( 250, 380 ] 19 0.033% 99.329% ( 380, 580 ] 39 0.067% 99.396% ( 580, 870 ] 65 0.112% 99.508% ( 870, 1300 ] 96 0.166% 99.674% ( 1300, 1900 ] 74 0.128% 99.802% ( 1900, 2900 ] 36 0.062% 99.864% ( 2900, 4400 ] 58 0.100% 99.964% ( 4400, 6600 ] 16 0.028% 99.991% ( 6600, 9900 ] 4 0.007% 99.998% ( 9900, 14000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 163202 Average: 12.5942 StdDev: 143.33 Min: 1 Median: 1.8984 Max: 9292 Percentiles: P50: 1.90 P75: 2.97 P99: 13.41 P99.9: 1649.37 P99.99: 2433.44 ------------------------------------------------------ [ 0, 1 ] 6722 4.119% 4.119% # ( 1, 2 ] 83343 51.067% 55.186% ########## ( 2, 3 ] 33483 20.516% 75.703% #### ( 3, 4 ] 16109 9.871% 85.573% ## ( 4, 6 ] 17029 10.434% 96.007% ## ( 6, 10 ] 4206 2.577% 98.585% # ( 10, 15 ] 993 0.608% 99.193% ( 15, 22 ] 157 0.096% 99.289% ( 22, 34 ] 11 0.007% 99.296% ( 34, 51 ] 5 0.003% 99.299% ( 51, 76 ] 4 0.002% 99.301% ( 76, 110 ] 3 0.002% 99.303% ( 110, 170 ] 28 0.017% 99.320% ( 170, 250 ] 44 0.027% 99.347% ( 250, 380 ] 104 0.064% 99.411% ( 380, 580 ] 184 0.113% 99.524% ( 580, 870 ] 257 0.157% 99.681% ( 870, 1300 ] 245 0.150% 99.831% ( 1300, 1900 ] 192 0.118% 99.949% ( 1900, 2900 ] 125 0.077% 100.026% ( 2900, 4400 ] 79 0.048% 100.074% ( 4400, 6600 ] 22 0.013% 100.088% ( 6600, 9900 ] 2 0.001% 100.089% ** Level 3 read latency histogram (micros): Count: 686804 Average: 11.3000 StdDev: 191.01 Min: 1 Median: 1.9187 Max: 54942 Percentiles: P50: 1.92 P75: 3.02 P99: 9.96 P99.9: 1245.00 P99.99: 1844.92 ------------------------------------------------------ [ 0, 1 ] 26877 3.913% 3.913% # ( 1, 2 ] 344529 50.164% 54.077% ########## ( 2, 3 ] 142571 20.759% 74.836% #### ( 3, 4 ] 73518 10.704% 85.540% ## ( 4, 6 ] 76107 11.081% 96.622% ## ( 6, 10 ] 16519 2.405% 99.027% ( 10, 15 ] 3144 0.458% 99.485% ( 15, 22 ] 392 0.057% 99.542% ( 22, 34 ] 40 0.006% 99.548% ( 34, 51 ] 39 0.006% 99.553% ( 51, 76 ] 34 0.005% 99.558% ( 76, 110 ] 15 0.002% 99.560% ( 110, 170 ] 60 0.009% 99.569% ( 170, 250 ] 101 0.015% 99.584% ( 250, 380 ] 248 0.036% 99.620% ( 380, 580 ] 457 0.067% 99.687% ( 580, 870 ] 740 0.108% 99.794% ( 870, 1300 ] 835 0.122% 99.916% ( 1300, 1900 ] 563 0.082% 99.998% ( 1900, 2900 ] 447 0.065% 100.063% ( 2900, 4400 ] 317 0.046% 100.109% ( 4400, 6600 ] 94 0.014% 100.123% ( 6600, 9900 ] 17 0.002% 100.125% ( 9900, 14000 ] 1 0.000% 100.125% ( 14000, 22000 ] 2 0.000% 100.126% ( 22000, 33000 ] 5 0.001% 100.126% ( 33000, 50000 ] 2 0.000% 100.127% ( 50000, 75000 ] 1 0.000% 100.127% ** Level 4 read latency histogram (micros): Count: 444647 Average: 23.1229 StdDev: 269.96 Min: 1 Median: 1.9376 Max: 44760 Percentiles: P50: 1.94 P75: 3.09 P99: 464.10 P99.9: 2632.27 P99.99: 3482.40 ------------------------------------------------------ [ 0, 1 ] 15668 3.524% 3.524% # ( 1, 2 ] 220409 49.569% 53.093% ########## ( 2, 3 ] 92890 20.891% 73.984% #### ( 3, 4 ] 48004 10.796% 84.780% ## ( 4, 6 ] 48510 10.910% 95.690% ## ( 6, 10 ] 11573 2.603% 98.292% # ( 10, 15 ] 2126 0.478% 98.770% ( 15, 22 ] 191 0.043% 98.813% ( 22, 34 ] 23 0.005% 98.819% ( 34, 51 ] 27 0.006% 98.825% ( 51, 76 ] 46 0.010% 98.835% ( 76, 110 ] 19 0.004% 98.839% ( 110, 170 ] 57 0.013% 98.852% ( 170, 250 ] 143 0.032% 98.884% ( 250, 380 ] 261 0.059% 98.943% ( 380, 580 ] 610 0.137% 99.080% ( 580, 870 ] 1024 0.230% 99.310% ( 870, 1300 ] 1256 0.282% 99.593% ( 1300, 1900 ] 877 0.197% 99.790% ( 1900, 2900 ] 671 0.151% 99.941% ( 2900, 4400 ] 568 0.128% 100.069% ( 4400, 6600 ] 193 0.043% 100.112% ( 6600, 9900 ] 20 0.004% 100.117% ( 9900, 14000 ] 1 0.000% 100.117% ( 14000, 22000 ] 5 0.001% 100.118% ( 22000, 33000 ] 6 0.001% 100.119% ( 33000, 50000 ] 3 0.001% 100.120% ** Level 5 read latency histogram (micros): Count: 12411 Average: 1747.4396 StdDev: 1758.18 Min: 94 Median: 1203.5554 Max: 65555 Percentiles: P50: 1203.56 P75: 2359.46 P99: 7341.11 P99.9: 10985.82 P99.99: 48633.77 ------------------------------------------------------ ( 76, 110 ] 3 0.024% 0.024% ( 110, 170 ] 54 0.435% 0.459% ( 170, 250 ] 168 1.354% 1.813% ( 250, 380 ] 537 4.327% 6.140% # ( 380, 580 ] 1271 10.241% 16.381% ## ( 580, 870 ] 2234 18.000% 34.381% #### ( 870, 1300 ] 2499 20.135% 54.516% #### ( 1300, 1900 ] 1848 14.890% 69.406% ### ( 1900, 2900 ] 1511 12.175% 81.581% ## ( 2900, 4400 ] 1463 11.788% 93.369% ## ( 4400, 6600 ] 667 5.374% 98.743% # ( 6600, 9900 ] 142 1.144% 99.887% ( 9900, 14000 ] 6 0.048% 99.936% ( 14000, 22000 ] 1 0.008% 99.944% ( 22000, 33000 ] 3 0.024% 99.968% ( 33000, 50000 ] 3 0.024% 99.992% ( 50000, 75000 ] 1 0.008% 100.000% ** DB Stats ** Uptime(secs): 7.8 total, 7.6 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 125.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.669 H:M:S, 46.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 128.82 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.669 H:M:S, 48.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/12/15-13:44:00 ... thread 0: (1000000,2000000) ops and (235493.8,168305.5) ops/second in (4.246397,11.883155) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/11 2.86 GB 163.6 1.5 0.0 1.5 3.4 1.8 0.0 1.8 80.7 177.5 19.51 11.54 30 0.650 1599K 1878 0.0 0.0 L1 7/7 830.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 83.61 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 17/0 954.57 MB 1.0 2.4 1.5 1.0 2.4 1.4 0.0 1.7 104.5 103.5 23.80 11.25 10 2.380 820K 0 0.0 0.0 L4 173/0 9.66 GB 1.0 6.3 3.3 3.0 6.2 3.2 0.0 1.9 119.4 117.5 53.85 29.53 42 1.282 2254K 0 0.0 0.0 L5 1140/0 93.63 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1355/18 107.98 GB 0.0 10.2 4.7 5.5 12.0 6.5 3.0 6.5 108.0 126.1 97.17 52.32 82 1.185 4673K 1878 0.0 0.0 Int 0/0 0.00 KB 0.0 10.2 4.7 5.5 12.0 6.5 3.0 6.5 108.0 126.1 97.17 52.32 82 1.185 4673K 1878 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.2 4.7 5.5 10.1 4.6 0.0 0.0 122.1 120.6 85.95 46.99 57 1.508 4673K 1878 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 168.4 11.22 5.34 25 0.449 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.1 total, 11.9 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.97 GB write, 1015.46 MB/s write, 10.24 GB read, 869.32 MB/s read, 97.2 seconds Interval compaction: 11.97 GB write, 1031.02 MB/s write, 10.24 GB read, 882.64 MB/s read, 97.2 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 42, interval: 42 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 619478 Average: 15.6293 StdDev: 157.42 Min: 1 Median: 1.7247 Max: 45624 Percentiles: P50: 1.72 P75: 2.65 P99: 314.98 P99.9: 1818.12 P99.99: 3145.26 ------------------------------------------------------ [ 0, 1 ] 69138 11.161% 11.161% ## ( 1, 2 ] 332015 53.596% 64.757% ########### ( 2, 3 ] 97770 15.783% 80.539% ### ( 3, 4 ] 47777 7.712% 88.252% ## ( 4, 6 ] 46684 7.536% 95.788% ## ( 6, 10 ] 9782 1.579% 97.367% ( 10, 15 ] 1658 0.268% 97.634% ( 15, 22 ] 181 0.029% 97.664% ( 22, 34 ] 23 0.004% 97.667% ( 34, 51 ] 18 0.003% 97.670% ( 51, 76 ] 23 0.004% 97.674% ( 76, 110 ] 951 0.154% 97.828% ( 110, 170 ] 3997 0.645% 98.473% ( 170, 250 ] 2455 0.396% 98.869% ( 250, 380 ] 1623 0.262% 99.131% ( 380, 580 ] 1800 0.291% 99.422% ( 580, 870 ] 1452 0.234% 99.656% ( 870, 1300 ] 977 0.158% 99.814% ( 1300, 1900 ] 619 0.100% 99.914% ( 1900, 2900 ] 424 0.068% 99.982% ( 2900, 4400 ] 300 0.048% 100.031% ( 4400, 6600 ] 67 0.011% 100.041% ( 6600, 9900 ] 12 0.002% 100.043% ( 9900, 14000 ] 2 0.000% 100.044% ( 14000, 22000 ] 1 0.000% 100.044% ( 33000, 50000 ] 1 0.000% 100.044% ** Level 1 read latency histogram (micros): Count: 58297 Average: 17.7507 StdDev: 196.49 Min: 1 Median: 1.9929 Max: 10779 Percentiles: P50: 1.99 P75: 3.32 P99: 208.61 P99.9: 3507.45 P99.99: 6492.63 ------------------------------------------------------ [ 0, 1 ] 1911 3.278% 3.278% # ( 1, 2 ] 27432 47.056% 50.334% ######### ( 2, 3 ] 12409 21.286% 71.619% #### ( 3, 4 ] 6243 10.709% 82.328% ## ( 4, 6 ] 7122 12.217% 94.545% ## ( 6, 10 ] 1927 3.305% 97.851% # ( 10, 15 ] 398 0.683% 98.533% ( 15, 22 ] 63 0.108% 98.641% ( 22, 34 ] 18 0.031% 98.672% ( 34, 51 ] 2 0.003% 98.676% ( 51, 76 ] 3 0.005% 98.681% ( 76, 110 ] 41 0.070% 98.751% ( 110, 170 ] 118 0.202% 98.954% ( 170, 250 ] 56 0.096% 99.050% ( 250, 380 ] 62 0.106% 99.156% ( 380, 580 ] 80 0.137% 99.293% ( 580, 870 ] 94 0.161% 99.455% ( 870, 1300 ] 111 0.190% 99.645% ( 1300, 1900 ] 83 0.142% 99.787% ( 1900, 2900 ] 41 0.070% 99.858% ( 2900, 4400 ] 61 0.105% 99.962% ( 4400, 6600 ] 17 0.029% 99.991% ( 6600, 9900 ] 4 0.007% 99.998% ( 9900, 14000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 163242 Average: 12.6787 StdDev: 143.53 Min: 1 Median: 1.8987 Max: 9292 Percentiles: P50: 1.90 P75: 2.97 P99: 13.61 P99.9: 1652.37 P99.99: 2437.11 ------------------------------------------------------ [ 0, 1 ] 6722 4.118% 4.118% # ( 1, 2 ] 83343 51.055% 55.173% ########## ( 2, 3 ] 33483 20.511% 75.684% #### ( 3, 4 ] 16109 9.868% 85.552% ## ( 4, 6 ] 17029 10.432% 95.984% ## ( 6, 10 ] 4206 2.577% 98.560% # ( 10, 15 ] 993 0.608% 99.169% ( 15, 22 ] 157 0.096% 99.265% ( 22, 34 ] 11 0.007% 99.272% ( 34, 51 ] 5 0.003% 99.275% ( 51, 76 ] 4 0.002% 99.277% ( 76, 110 ] 5 0.003% 99.280% ( 110, 170 ] 41 0.025% 99.305% ( 170, 250 ] 56 0.034% 99.340% ( 250, 380 ] 106 0.065% 99.405% ( 380, 580 ] 188 0.115% 99.520% ( 580, 870 ] 259 0.159% 99.678% ( 870, 1300 ] 249 0.153% 99.831% ( 1300, 1900 ] 192 0.118% 99.949% ( 1900, 2900 ] 126 0.077% 100.026% ( 2900, 4400 ] 79 0.048% 100.074% ( 4400, 6600 ] 22 0.013% 100.088% ( 6600, 9900 ] 2 0.001% 100.089% ** Level 3 read latency histogram (micros): Count: 877515 Average: 10.3739 StdDev: 176.71 Min: 1 Median: 1.9052 Max: 54942 Percentiles: P50: 1.91 P75: 2.97 P99: 9.90 P99.9: 1159.75 P99.99: 1769.85 ------------------------------------------------------ [ 0, 1 ] 34066 3.882% 3.882% # ( 1, 2 ] 447069 50.947% 54.829% ########## ( 2, 3 ] 182985 20.853% 75.682% #### ( 3, 4 ] 90618 10.327% 86.009% ## ( 4, 6 ] 93385 10.642% 96.651% ## ( 6, 10 ] 21126 2.407% 99.058% ( 10, 15 ] 3865 0.440% 99.498% ( 15, 22 ] 478 0.054% 99.553% ( 22, 34 ] 47 0.005% 99.558% ( 34, 51 ] 53 0.006% 99.564% ( 51, 76 ] 57 0.006% 99.571% ( 76, 110 ] 52 0.006% 99.577% ( 110, 170 ] 211 0.024% 99.601% ( 170, 250 ] 210 0.024% 99.625% ( 250, 380 ] 336 0.038% 99.663% ( 380, 580 ] 571 0.065% 99.728% ( 580, 870 ] 867 0.099% 99.827% ( 870, 1300 ] 952 0.108% 99.935% ( 1300, 1900 ] 612 0.070% 100.005% ( 1900, 2900 ] 483 0.055% 100.060% ( 2900, 4400 ] 357 0.041% 100.101% ( 4400, 6600 ] 104 0.012% 100.113% ( 6600, 9900 ] 25 0.003% 100.116% ( 9900, 14000 ] 4 0.000% 100.116% ( 14000, 22000 ] 3 0.000% 100.116% ( 22000, 33000 ] 5 0.001% 100.117% ( 33000, 50000 ] 2 0.000% 100.117% ( 50000, 75000 ] 1 0.000% 100.117% ** Level 4 read latency histogram (micros): Count: 784377 Average: 18.3375 StdDev: 220.11 Min: 1 Median: 1.9071 Max: 44760 Percentiles: P50: 1.91 P75: 2.97 P99: 224.98 P99.9: 1873.27 P99.99: 2726.52 ------------------------------------------------------ [ 0, 1 ] 27271 3.477% 3.477% # ( 1, 2 ] 402303 51.289% 54.766% ########## ( 2, 3 ] 163986 20.907% 75.673% #### ( 3, 4 ] 78699 10.033% 85.706% ## ( 4, 6 ] 78528 10.012% 95.718% ## ( 6, 10 ] 19390 2.472% 98.190% ( 10, 15 ] 3471 0.443% 98.632% ( 15, 22 ] 311 0.040% 98.672% ( 22, 34 ] 41 0.005% 98.677% ( 34, 51 ] 49 0.006% 98.683% ( 51, 76 ] 71 0.009% 98.692% ( 76, 110 ] 355 0.045% 98.738% ( 110, 170 ] 1384 0.176% 98.914% ( 170, 250 ] 981 0.125% 99.039% ( 250, 380 ] 945 0.120% 99.160% ( 380, 580 ] 1373 0.175% 99.335% ( 580, 870 ] 1731 0.221% 99.555% ( 870, 1300 ] 1709 0.218% 99.773% ( 1300, 1900 ] 1041 0.133% 99.906% ( 1900, 2900 ] 798 0.102% 100.008% ( 2900, 4400 ] 692 0.088% 100.096% ( 4400, 6600 ] 220 0.028% 100.124% ( 6600, 9900 ] 26 0.003% 100.127% ( 9900, 14000 ] 4 0.001% 100.128% ( 14000, 22000 ] 6 0.001% 100.129% ( 22000, 33000 ] 6 0.001% 100.129% ( 33000, 50000 ] 3 0.000% 100.130% ** Level 5 read latency histogram (micros): Count: 41726 Average: 798.8418 StdDev: 1240.61 Min: 78 Median: 368.5324 Max: 65555 Percentiles: P50: 368.53 P75: 894.23 P99: 5843.14 P99.9: 9450.03 P99.99: 28700.47 ------------------------------------------------------ ( 76, 110 ] 2531 6.066% 6.066% # ( 110, 170 ] 9335 22.372% 28.438% #### ( 170, 250 ] 5462 13.090% 41.528% ### ( 250, 380 ] 3877 9.292% 50.820% ## ( 380, 580 ] 4978 11.930% 62.750% ## ( 580, 870 ] 4895 11.731% 74.481% ## ( 870, 1300 ] 3842 9.208% 83.689% ## ( 1300, 1900 ] 2317 5.553% 89.242% # ( 1900, 2900 ] 1876 4.496% 93.738% # ( 2900, 4400 ] 1730 4.146% 97.884% # ( 4400, 6600 ] 710 1.702% 99.585% ( 6600, 9900 ] 152 0.364% 99.950% ( 9900, 14000 ] 8 0.019% 99.969% ( 14000, 22000 ] 7 0.017% 99.986% ( 22000, 33000 ] 3 0.007% 99.993% ( 33000, 50000 ] 3 0.007% 100.000% ( 50000, 75000 ] 1 0.002% 100.002% ** DB Stats ** Uptime(secs): 12.1 total, 4.2 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 163.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.322 H:M:S, 35.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 231.78 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.653 H:M:S, 15.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/12/15-13:44:06 ... thread 0: (1000000,3000000) ops and (166677.3,167759.2) ops/second in (5.999616,17.882771) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/14 3.85 GB 132.1 2.8 0.0 2.8 5.6 2.8 0.0 2.0 104.0 210.1 27.27 18.82 51 0.535 2878K 3102 0.0 0.0 L1 7/7 830.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/0 83.61 MB 0.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 17/0 954.57 MB 1.0 2.4 1.5 1.0 2.4 1.4 0.0 1.7 104.5 103.5 23.80 11.25 10 2.380 820K 0 0.0 0.0 L4 173/0 9.66 GB 1.0 6.3 3.3 3.0 6.2 3.2 0.0 1.9 119.4 117.5 53.85 29.53 42 1.282 2254K 0 0.0 0.0 L5 1140/0 93.63 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1356/21 108.96 GB 0.0 11.5 4.7 6.7 14.2 7.4 3.0 5.0 112.0 138.4 104.92 59.60 103 1.019 5953K 3102 0.0 0.0 Int 0/0 0.00 KB 0.0 11.5 4.7 6.7 14.2 7.4 3.0 5.0 112.0 138.4 104.92 59.60 103 1.019 5953K 3102 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 11.5 4.7 6.7 11.4 4.6 0.0 0.0 129.7 128.3 90.59 51.54 62 1.461 5953K 3102 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 202.1 14.34 8.06 41 0.350 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 18.1 total, 17.9 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 14.18 GB write, 803.72 MB/s write, 11.48 GB read, 650.39 MB/s read, 104.9 seconds Interval compaction: 14.18 GB write, 811.90 MB/s write, 11.48 GB read, 657.01 MB/s read, 104.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 7, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 7, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 42, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 7, total-stops: 42, interval: 49 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1114050 Average: 15.9929 StdDev: 124.65 Min: 0 Median: 1.5924 Max: 45624 Percentiles: P50: 1.59 P75: 2.28 P99: 271.85 P99.9: 1179.71 P99.99: 2069.46 ------------------------------------------------------ [ 0, 1 ] 205700 18.464% 18.464% #### ( 1, 2 ] 593056 53.234% 71.698% ########### ( 2, 3 ] 131540 11.807% 83.506% ## ( 3, 4 ] 62254 5.588% 89.094% # ( 4, 6 ] 54972 4.934% 94.028% # ( 6, 10 ] 11314 1.016% 95.044% ( 10, 15 ] 1944 0.174% 95.218% ( 15, 22 ] 215 0.019% 95.238% ( 22, 34 ] 35 0.003% 95.241% ( 34, 51 ] 19 0.002% 95.242% ( 51, 76 ] 40 0.004% 95.246% ( 76, 110 ] 5891 0.529% 95.775% ( 110, 170 ] 24633 2.211% 97.986% ( 170, 250 ] 10594 0.951% 98.937% ( 250, 380 ] 4191 0.376% 99.313% ( 380, 580 ] 3780 0.339% 99.652% ( 580, 870 ] 2008 0.180% 99.833% ( 870, 1300 ] 1044 0.094% 99.926% ( 1300, 1900 ] 634 0.057% 99.983% ( 1900, 2900 ] 452 0.041% 100.024% ( 2900, 4400 ] 309 0.028% 100.052% ( 4400, 6600 ] 67 0.006% 100.058% ( 6600, 9900 ] 12 0.001% 100.059% ( 9900, 14000 ] 2 0.000% 100.059% ( 14000, 22000 ] 1 0.000% 100.059% ( 33000, 50000 ] 1 0.000% 100.059% ** Level 1 read latency histogram (micros): Count: 59276 Average: 20.3603 StdDev: 196.35 Min: 1 Median: 2.0238 Max: 10779 Percentiles: P50: 2.02 P75: 3.43 P99: 330.99 P99.9: 3483.38 P99.99: 6479.96 ------------------------------------------------------ [ 0, 1 ] 1911 3.224% 3.224% # ( 1, 2 ] 27432 46.278% 49.502% ######### ( 2, 3 ] 12409 20.934% 70.437% #### ( 3, 4 ] 6243 10.532% 80.969% ## ( 4, 6 ] 7122 12.015% 92.984% ## ( 6, 10 ] 1927 3.251% 96.235% # ( 10, 15 ] 398 0.671% 96.906% ( 15, 22 ] 63 0.106% 97.012% ( 22, 34 ] 18 0.030% 97.043% ( 34, 51 ] 2 0.003% 97.046% ( 51, 76 ] 3 0.005% 97.051% ( 76, 110 ] 202 0.341% 97.392% ( 110, 170 ] 624 1.053% 98.445% ( 170, 250 ] 247 0.417% 98.861% ( 250, 380 ] 132 0.223% 99.084% ( 380, 580 ] 122 0.206% 99.290% ( 580, 870 ] 100 0.169% 99.458% ( 870, 1300 ] 114 0.192% 99.651% ( 1300, 1900 ] 83 0.140% 99.791% ( 1900, 2900 ] 41 0.069% 99.860% ( 2900, 4400 ] 61 0.103% 99.963% ( 4400, 6600 ] 17 0.029% 99.992% ( 6600, 9900 ] 4 0.007% 99.998% ( 9900, 14000 ] 1 0.002% 100.000% ** Level 2 read latency histogram (micros): Count: 163332 Average: 12.7770 StdDev: 143.58 Min: 1 Median: 1.8992 Max: 9292 Percentiles: P50: 1.90 P75: 2.97 P99: 14.06 P99.9: 1652.09 P99.99: 2437.04 ------------------------------------------------------ [ 0, 1 ] 6722 4.116% 4.116% # ( 1, 2 ] 83343 51.027% 55.142% ########## ( 2, 3 ] 33483 20.500% 75.642% #### ( 3, 4 ] 16109 9.863% 85.505% ## ( 4, 6 ] 17029 10.426% 95.931% ## ( 6, 10 ] 4206 2.575% 98.506% # ( 10, 15 ] 993 0.608% 99.114% ( 15, 22 ] 157 0.096% 99.210% ( 22, 34 ] 11 0.007% 99.217% ( 34, 51 ] 5 0.003% 99.220% ( 51, 76 ] 4 0.002% 99.222% ( 76, 110 ] 12 0.007% 99.230% ( 110, 170 ] 93 0.057% 99.287% ( 170, 250 ] 73 0.045% 99.331% ( 250, 380 ] 112 0.069% 99.400% ( 380, 580 ] 195 0.119% 99.519% ( 580, 870 ] 260 0.159% 99.679% ( 870, 1300 ] 249 0.152% 99.831% ( 1300, 1900 ] 192 0.118% 99.949% ( 1900, 2900 ] 126 0.077% 100.026% ( 2900, 4400 ] 79 0.048% 100.074% ( 4400, 6600 ] 22 0.013% 100.088% ( 6600, 9900 ] 2 0.001% 100.089% ** Level 3 read latency histogram (micros): Count: 878715 Average: 10.6113 StdDev: 176.77 Min: 1 Median: 1.9066 Max: 54942 Percentiles: P50: 1.91 P75: 2.97 P99: 10.88 P99.9: 1159.80 P99.99: 1770.71 ------------------------------------------------------ [ 0, 1 ] 34066 3.877% 3.877% # ( 1, 2 ] 447069 50.878% 54.754% ########## ( 2, 3 ] 182985 20.824% 75.579% #### ( 3, 4 ] 90618 10.313% 85.891% ## ( 4, 6 ] 93385 10.627% 96.519% ## ( 6, 10 ] 21126 2.404% 98.923% ( 10, 15 ] 3865 0.440% 99.363% ( 15, 22 ] 478 0.054% 99.417% ( 22, 34 ] 47 0.005% 99.422% ( 34, 51 ] 53 0.006% 99.428% ( 51, 76 ] 57 0.006% 99.435% ( 76, 110 ] 219 0.025% 99.460% ( 110, 170 ] 848 0.097% 99.556% ( 170, 250 ] 444 0.051% 99.607% ( 250, 380 ] 420 0.048% 99.655% ( 380, 580 ] 626 0.071% 99.726% ( 580, 870 ] 888 0.101% 99.827% ( 870, 1300 ] 953 0.108% 99.935% ( 1300, 1900 ] 612 0.070% 100.005% ( 1900, 2900 ] 484 0.055% 100.060% ( 2900, 4400 ] 357 0.041% 100.101% ( 4400, 6600 ] 104 0.012% 100.113% ( 6600, 9900 ] 25 0.003% 100.115% ( 9900, 14000 ] 4 0.000% 100.116% ( 14000, 22000 ] 3 0.000% 100.116% ( 22000, 33000 ] 5 0.001% 100.117% ( 33000, 50000 ] 2 0.000% 100.117% ( 50000, 75000 ] 1 0.000% 100.117% ** Level 4 read latency histogram (micros): Count: 797256 Average: 20.6270 StdDev: 219.57 Min: 1 Median: 1.9228 Max: 44760 Percentiles: P50: 1.92 P75: 3.05 P99: 323.99 P99.9: 1867.09 P99.99: 2725.34 ------------------------------------------------------ [ 0, 1 ] 27271 3.421% 3.421% # ( 1, 2 ] 402414 50.475% 53.895% ########## ( 2, 3 ] 164387 20.619% 74.515% #### ( 3, 4 ] 78953 9.903% 84.418% ## ( 4, 6 ] 78991 9.908% 94.326% ## ( 6, 10 ] 19441 2.438% 96.764% ( 10, 15 ] 3485 0.437% 97.201% ( 15, 22 ] 312 0.039% 97.240% ( 22, 34 ] 41 0.005% 97.245% ( 34, 51 ] 49 0.006% 97.252% ( 51, 76 ] 71 0.009% 97.260% ( 76, 110 ] 2129 0.267% 97.528% ( 110, 170 ] 7600 0.953% 98.481% ( 170, 250 ] 3202 0.402% 98.882% ( 250, 380 ] 1647 0.207% 99.089% ( 380, 580 ] 1890 0.237% 99.326% ( 580, 870 ] 1860 0.233% 99.559% ( 870, 1300 ] 1729 0.217% 99.776% ( 1300, 1900 ] 1044 0.131% 99.907% ( 1900, 2900 ] 800 0.100% 100.008% ( 2900, 4400 ] 692 0.087% 100.094% ( 4400, 6600 ] 220 0.028% 100.122% ( 6600, 9900 ] 26 0.003% 100.125% ( 9900, 14000 ] 4 0.001% 100.126% ( 14000, 22000 ] 6 0.001% 100.126% ( 22000, 33000 ] 6 0.001% 100.127% ( 33000, 50000 ] 3 0.000% 100.128% ** Level 5 read latency histogram (micros): Count: 124128 Average: 388.6393 StdDev: 782.10 Min: 75 Median: 163.3187 Max: 65555 Percentiles: P50: 163.32 P75: 306.57 P99: 4088.85 P99.9: 7639.33 P99.99: 13788.44 ------------------------------------------------------ ( 51, 76 ] 5 0.004% 0.004% ( 76, 110 ] 14768 11.897% 11.901% ## ( 110, 170 ] 53217 42.873% 54.774% ######### ( 170, 250 ] 21182 17.065% 71.839% ### ( 250, 380 ] 9017 7.264% 79.103% # ( 380, 580 ] 9142 7.365% 86.468% # ( 580, 870 ] 5991 4.826% 91.294% # ( 870, 1300 ] 3959 3.189% 94.484% # ( 1300, 1900 ] 2340 1.885% 96.369% ( 1900, 2900 ] 1893 1.525% 97.894% ( 2900, 4400 ] 1732 1.395% 99.289% ( 4400, 6600 ] 710 0.572% 99.861% ( 6600, 9900 ] 152 0.122% 99.984% ( 9900, 14000 ] 8 0.006% 99.990% ( 14000, 22000 ] 7 0.006% 99.996% ( 22000, 33000 ] 3 0.002% 99.998% ( 33000, 50000 ] 3 0.002% 100.001% ( 50000, 75000 ] 1 0.001% 100.002% ** DB Stats ** Uptime(secs): 18.1 total, 6.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 163.42 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:6.747 H:M:S, 37.3 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 164.04 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:2.425 H:M:S, 40.4 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 222.176 micros/op 17989 ops/sec 60.090 seconds 1080996 operations; 17.4 MB/s (261999 of 261999 found) Microseconds per read: Count: 1080996 Average: 222.1797 StdDev: 670.43 Min: 2 Median: 164.4029 Max: 117255 Percentiles: P50: 164.40 P75: 221.90 P99: 940.39 P99.9: 10401.89 P99.99: 29492.89 ------------------------------------------------------ ( 1, 2 ] 6 0.001% 0.001% ( 2, 3 ] 29 0.003% 0.003% ( 3, 4 ] 62 0.006% 0.009% ( 4, 6 ] 85 0.008% 0.017% ( 6, 10 ] 384 0.036% 0.052% ( 10, 15 ] 3602 0.333% 0.386% ( 15, 22 ] 4011 0.371% 0.757% ( 22, 34 ] 10255 0.949% 1.705% ( 34, 51 ] 2252 0.208% 1.914% ( 51, 76 ] 250 0.023% 1.937% ( 76, 110 ] 29034 2.686% 4.623% # ( 110, 170 ] 540995 50.046% 54.669% ########## ( 170, 250 ] 338781 31.340% 86.008% ###### ( 250, 380 ] 93042 8.607% 94.615% ## ( 380, 580 ] 33928 3.139% 97.754% # ( 580, 870 ] 12645 1.170% 98.924% ( 870, 1300 ] 5040 0.466% 99.390% ( 1300, 1900 ] 2257 0.209% 99.599% ( 1900, 2900 ] 1593 0.147% 99.746% ( 2900, 4400 ] 1065 0.099% 99.845% ( 4400, 6600 ] 347 0.032% 99.877% ( 6600, 9900 ] 199 0.018% 99.895% ( 9900, 14000 ] 433 0.040% 99.935% ( 14000, 22000 ] 488 0.045% 99.980% ( 22000, 33000 ] 154 0.014% 99.995% ( 33000, 50000 ] 45 0.004% 99.999% ( 50000, 75000 ] 13 0.001% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14425639 rocksdb.bloom.filter.full.positive COUNT : 1221119 rocksdb.bloom.filter.full.true.positive COUNT : 1080606 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 390 rocksdb.memtable.miss COUNT : 1080606 rocksdb.l0.hit COUNT : 37902 rocksdb.l1.hit COUNT : 16384 rocksdb.l2andup.hit COUNT : 1026320 rocksdb.compaction.key.drop.new COUNT : 183454 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3496775 rocksdb.number.keys.read COUNT : 1080996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3608671800 rocksdb.bytes.read COUNT : 1080996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1575 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 46303058 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 17821971229 rocksdb.compact.write.bytes COUNT : 17629287485 rocksdb.flush.write.bytes COUNT : 3567163699 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 26 rocksdb.number.superversion_acquires COUNT : 707 rocksdb.number.superversion_releases COUNT : 690 rocksdb.number.superversion_cleanups COUNT : 177 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 213 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3538339840 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1318912 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21735648727 rocksdb.non.last.level.read.count COUNT : 5108491 rocksdb.block.checksum.compute.count COUNT : 5547496 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 163.589479 P95 : 402.766235 P99 : 939.347226 P100 : 117254.000000 COUNT : 1080996 SUM : 239268895 rocksdb.db.write.micros P50 : 1.571474 P95 : 4.980657 P99 : 14.844290 P100 : 922255.000000 COUNT : 3496775 SUM : 55500743 rocksdb.compaction.times.micros P50 : 1357500.000000 P95 : 3425000.000000 P99 : 25850000.000000 P100 : 27798859.000000 COUNT : 65 SUM : 125120596 rocksdb.compaction.times.cpu_micros P50 : 790652.173913 P95 : 1802777.777778 P99 : 11270000.000000 P100 : 11513518.000000 COUNT : 65 SUM : 69681766 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 808.653846 P95 : 143000.000000 P99 : 472283.000000 P100 : 472283.000000 COUNT : 49 SUM : 1281544 rocksdb.compaction.outfile.sync.micros P50 : 53125.000000 P95 : 246857.142857 P99 : 469775.000000 P100 : 561971.000000 COUNT : 211 SUM : 15838413 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 500.000000 P95 : 4376.923077 P99 : 6145.333333 P100 : 6560.000000 COUNT : 124 SUM : 149143 rocksdb.table.open.io.micros P50 : 9525.660793 P95 : 20008.196721 P99 : 23546.875000 P100 : 87911.000000 COUNT : 1575 SUM : 15105581 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.960655 P95 : 81.441873 P99 : 212.883143 P100 : 54947.000000 COUNT : 4315522 SUM : 76827573 rocksdb.read.block.get.micros P50 : 138.565805 P95 : 295.574432 P99 : 868.874595 P100 : 65559.000000 COUNT : 1230437 SUM : 204298045 rocksdb.write.raw.block.micros P50 : 0.543077 P95 : 1.510940 P99 : 3.588940 P100 : 74532.000000 COUNT : 5134249 SUM : 14932704 rocksdb.numfiles.in.singlecompaction P50 : 2.660714 P95 : 5.625000 P99 : 25.000000 P100 : 29.000000 COUNT : 75 SUM : 278 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.576657 P95 : 1992.596567 P99 : 2719.206009 P100 : 922090.000000 COUNT : 211625 SUM : 46321617 rocksdb.sst.read.micros P50 : 2.412499 P95 : 169.420296 P99 : 469.502276 P100 : 65555.000000 COUNT : 5108590 SUM : 263841401 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1080996 SUM : 1080996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3496775 SUM : 3608671800 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 224444.444444 P95 : 1033400.000000 P99 : 1166680.000000 P100 : 1191802.000000 COUNT : 49 SUM : 15938458 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 220 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0