[2024-11-27 07:16:50.542036] Starting SPDK v25.01-pre git sha1 2f2acf4eb / DPDK 24.11.0-rc4 initialization... [2024-11-27 07:16:50.542364] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid383956 ] Set seed to 1732688210534876 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-27 07:16:50.731121] pci_dpdk.c: 37:dpdk_pci_init: *NOTICE*: In-development DPDK 24.11.0-rc4 is used. There is no support for it in SPDK. Enabled only for validation. [2024-11-27 07:16:50.790009] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-27 07:16:50.817041] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-27 07:16:50.817089] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 383956' to capture a snapshot of events at runtime. [2024-11-27 07:16:50.817104] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-27 07:16:50.817116] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-27 07:16:50.817128] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid383956 for offline analysis/debug. [2024-11-27 07:16:50.817683] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 27 07:16:55 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/11/27-07:17:03 ... thread 0: (1000000,1000000) ops and (123265.4,123265.4) ops/second in (8.112577,8.112577) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 2.39 GB 63.0 0.3 0.0 0.3 1.2 0.9 0.0 1.4 38.1 144.8 8.26 4.17 13 0.636 319K 409 0.0 0.0 L1 9/9 1.02 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 21/1 1.07 GB 1.0 1.3 0.6 0.6 1.3 0.6 0.0 2.0 114.2 113.2 11.36 6.44 5 2.271 392K 0 0.0 0.0 L4 172/2 9.66 GB 1.0 5.3 3.0 2.3 5.2 2.9 0.0 1.7 117.3 115.2 46.44 27.67 31 1.498 1330K 0 0.0 0.0 L5 1132/0 93.01 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1351/24 107.16 GB 0.0 6.9 3.6 3.3 7.6 4.4 3.0 8.9 106.9 118.6 66.06 38.28 49 1.348 2042K 409 0.0 0.0 Int 0/0 0.00 KB 0.0 6.9 3.6 3.3 7.6 4.4 3.0 8.9 106.9 118.6 66.06 38.28 49 1.348 2042K 409 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.9 3.6 3.3 6.8 3.5 0.0 0.0 118.2 116.4 59.72 35.48 37 1.614 2042K 409 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 139.0 6.34 2.80 12 0.529 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 8.3 total, 8.1 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.65 GB write, 947.20 MB/s write, 6.89 GB read, 853.86 MB/s read, 66.1 seconds Interval compaction: 7.65 GB write, 965.16 MB/s write, 6.89 GB read, 870.05 MB/s read, 66.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 18, interval: 32 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 372049 Average: 14.1991 StdDev: 154.08 Min: 1 Median: 2.0493 Max: 8646 Percentiles: P50: 2.05 P75: 2.97 P99: 179.71 P99.9: 2242.00 P99.99: 4107.73 ------------------------------------------------------ [ 0, 1 ] 14116 3.794% 3.794% # ( 1, 2 ] 166908 44.862% 48.656% ######### ( 2, 3 ] 101489 27.278% 75.934% ##### ( 3, 4 ] 36015 9.680% 85.615% ## ( 4, 6 ] 35732 9.604% 95.219% ## ( 6, 10 ] 10058 2.703% 97.922% # ( 10, 15 ] 2032 0.546% 98.468% ( 15, 22 ] 287 0.077% 98.545% ( 22, 34 ] 29 0.008% 98.553% ( 34, 51 ] 15 0.004% 98.557% ( 51, 76 ] 14 0.004% 98.561% ( 76, 110 ] 329 0.088% 98.649% ( 110, 170 ] 1214 0.326% 98.976% ( 170, 250 ] 754 0.203% 99.178% ( 250, 380 ] 510 0.137% 99.315% ( 380, 580 ] 607 0.163% 99.479% ( 580, 870 ] 538 0.145% 99.623% ( 870, 1300 ] 570 0.153% 99.776% ( 1300, 1900 ] 384 0.103% 99.880% ( 1900, 2900 ] 225 0.060% 99.940% ( 2900, 4400 ] 232 0.062% 100.002% ( 4400, 6600 ] 66 0.018% 100.020% ( 6600, 9900 ] 14 0.004% 100.024% ** Level 1 read latency histogram (micros): Count: 193 Average: 1079.0881 StdDev: 1911.11 Min: 80 Median: 344.2500 Max: 12944 Percentiles: P50: 344.25 P75: 1138.75 P99: 9388.50 P99.9: 12944.00 P99.99: 12944.00 ------------------------------------------------------ ( 76, 110 ] 9 4.663% 4.663% # ( 110, 170 ] 43 22.280% 26.943% #### ( 170, 250 ] 30 15.544% 42.487% ### ( 250, 380 ] 20 10.363% 52.850% ## ( 380, 580 ] 13 6.736% 59.585% # ( 580, 870 ] 21 10.881% 70.466% ## ( 870, 1300 ] 14 7.254% 77.720% # ( 1300, 1900 ] 17 8.808% 86.528% ## ( 1900, 2900 ] 10 5.181% 91.710% # ( 2900, 4400 ] 5 2.591% 94.301% # ( 4400, 6600 ] 4 2.073% 96.373% ( 6600, 9900 ] 6 3.109% 99.482% # ( 9900, 14000 ] 1 0.518% 100.000% ** Level 2 read latency histogram (micros): Count: 65435 Average: 9.1620 StdDev: 129.53 Min: 1 Median: 2.0190 Max: 6791 Percentiles: P50: 2.02 P75: 2.93 P99: 10.26 P99.9: 2218.83 P99.99: 5661.87 ------------------------------------------------------ [ 0, 1 ] 1560 2.384% 2.384% ( 1, 2 ] 30815 47.093% 49.477% ######### ( 2, 3 ] 18054 27.591% 77.067% ###### ( 3, 4 ] 6194 9.466% 86.533% ## ( 4, 6 ] 6268 9.579% 96.112% ## ( 6, 10 ] 1872 2.861% 98.973% # ( 10, 15 ] 345 0.527% 99.500% ( 15, 22 ] 34 0.052% 99.552% ( 22, 34 ] 12 0.018% 99.571% ( 34, 51 ] 15 0.023% 99.593% ( 51, 76 ] 12 0.018% 99.612% ( 76, 110 ] 7 0.011% 99.623% ( 110, 170 ] 7 0.011% 99.633% ( 170, 250 ] 12 0.018% 99.652% ( 250, 380 ] 14 0.021% 99.673% ( 380, 580 ] 20 0.031% 99.704% ( 580, 870 ] 37 0.057% 99.760% ( 870, 1300 ] 50 0.076% 99.836% ( 1300, 1900 ] 32 0.049% 99.885% ( 1900, 2900 ] 30 0.046% 99.931% ( 2900, 4400 ] 31 0.047% 99.979% ( 4400, 6600 ] 13 0.020% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 761964 Average: 11.1969 StdDev: 183.43 Min: 1 Median: 2.2948 Max: 49405 Percentiles: P50: 2.29 P75: 3.26 P99: 9.82 P99.9: 160.42 P99.99: 656.61 ------------------------------------------------------ [ 0, 1 ] 9346 1.227% 1.227% ( 1, 2 ] 301786 39.606% 40.833% ######## ( 2, 3 ] 236946 31.097% 71.930% ###### ( 3, 4 ] 89512 11.748% 83.677% ## ( 4, 6 ] 94547 12.408% 96.086% ## ( 6, 10 ] 23239 3.050% 99.135% # ( 10, 15 ] 4750 0.623% 99.759% ( 15, 22 ] 654 0.086% 99.845% ( 22, 34 ] 87 0.011% 99.856% ( 34, 51 ] 76 0.010% 99.866% ( 51, 76 ] 80 0.010% 99.877% ( 76, 110 ] 95 0.012% 99.889% ( 110, 170 ] 100 0.013% 99.902% ( 170, 250 ] 95 0.012% 99.915% ( 250, 380 ] 151 0.020% 99.934% ( 380, 580 ] 297 0.039% 99.973% ( 580, 870 ] 480 0.063% 100.036% ( 870, 1300 ] 579 0.076% 100.112% ( 1300, 1900 ] 478 0.063% 100.175% ( 1900, 2900 ] 442 0.058% 100.233% ( 2900, 4400 ] 459 0.060% 100.293% ( 4400, 6600 ] 157 0.021% 100.314% ( 6600, 9900 ] 44 0.006% 100.320% ( 9900, 14000 ] 9 0.001% 100.321% ( 14000, 22000 ] 8 0.001% 100.322% ( 33000, 50000 ] 3 0.000% 100.322% ** Level 4 read latency histogram (micros): Count: 563789 Average: 19.8431 StdDev: 288.12 Min: 1 Median: 2.2854 Max: 55105 Percentiles: P50: 2.29 P75: 3.25 P99: 15.11 P99.9: 1695.13 P99.99: 2282.77 ------------------------------------------------------ [ 0, 1 ] 7055 1.251% 1.251% ( 1, 2 ] 224937 39.897% 41.149% ######## ( 2, 3 ] 174845 31.012% 72.161% ###### ( 3, 4 ] 63292 11.226% 83.387% ## ( 4, 6 ] 67348 11.946% 95.333% ## ( 6, 10 ] 17231 3.056% 98.389% # ( 10, 15 ] 3435 0.609% 98.999% ( 15, 22 ] 496 0.088% 99.087% ( 22, 34 ] 56 0.010% 99.096% ( 34, 51 ] 34 0.006% 99.103% ( 51, 76 ] 73 0.013% 99.115% ( 76, 110 ] 151 0.027% 99.142% ( 110, 170 ] 561 0.100% 99.242% ( 170, 250 ] 444 0.079% 99.320% ( 250, 380 ] 435 0.077% 99.398% ( 380, 580 ] 548 0.097% 99.495% ( 580, 870 ] 742 0.132% 99.626% ( 870, 1300 ] 991 0.176% 99.802% ( 1300, 1900 ] 837 0.148% 99.951% ( 1900, 2900 ] 579 0.103% 100.053% ( 2900, 4400 ] 536 0.095% 100.148% ( 4400, 6600 ] 240 0.043% 100.191% ( 6600, 9900 ] 75 0.013% 100.204% ( 9900, 14000 ] 15 0.003% 100.207% ( 14000, 22000 ] 17 0.003% 100.210% ( 22000, 33000 ] 6 0.001% 100.211% ( 33000, 50000 ] 5 0.001% 100.212% ( 50000, 75000 ] 1 0.000% 100.212% ** Level 5 read latency histogram (micros): Count: 19645 Average: 1158.8431 StdDev: 2070.58 Min: 80 Median: 460.6957 Max: 71975 Percentiles: P50: 460.70 P75: 1368.49 P99: 7162.38 P99.9: 23301.67 P99.99: 67983.93 ------------------------------------------------------ ( 76, 110 ] 947 4.821% 4.821% # ( 110, 170 ] 3669 18.677% 23.497% #### ( 170, 250 ] 2573 13.097% 36.595% ### ( 250, 380 ] 1810 9.214% 45.808% ## ( 380, 580 ] 2041 10.389% 56.198% ## ( 580, 870 ] 1788 9.102% 65.299% ## ( 870, 1300 ] 1744 8.878% 74.177% ## ( 1300, 1900 ] 1417 7.213% 81.390% # ( 1900, 2900 ] 1205 6.134% 87.524% # ( 2900, 4400 ] 1415 7.203% 94.726% # ( 4400, 6600 ] 807 4.108% 98.834% # ( 6600, 9900 ] 191 0.972% 99.807% ( 9900, 14000 ] 15 0.076% 99.883% ( 14000, 22000 ] 3 0.015% 99.898% ( 22000, 33000 ] 3 0.015% 99.913% ( 33000, 50000 ] 10 0.051% 99.964% ( 50000, 75000 ] 7 0.036% 100.000% ** DB Stats ** Uptime(secs): 8.3 total, 8.1 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 119.02 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.375 H:M:S, 40.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 121.27 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.375 H:M:S, 41.6 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/11/27-07:17:07 ... thread 0: (1000000,2000000) ops and (215371.6,156792.5) ops/second in (4.643138,12.755715) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/8 3.38 GB 188.8 1.5 0.0 1.5 3.4 1.8 0.0 1.8 94.2 207.1 16.72 11.99 34 0.492 1599K 1633 0.0 0.0 L1 9/9 1.02 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/0 991.13 MB 1.0 1.3 0.6 0.6 1.3 0.6 0.0 2.0 114.2 113.2 11.36 6.44 5 2.271 392K 0 0.0 0.0 L4 174/0 9.76 GB 1.0 5.5 3.1 2.4 5.4 3.0 0.0 1.7 119.2 117.0 47.49 28.64 32 1.484 1489K 0 0.0 0.0 L5 1132/0 93.01 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1353/17 108.14 GB 0.0 8.3 3.7 4.6 10.1 5.5 3.0 5.5 112.9 136.4 75.57 47.07 71 1.064 3480K 1633 0.0 0.0 Int 0/0 0.00 KB 0.0 8.3 3.7 4.6 10.1 5.5 3.0 5.5 112.9 136.4 75.57 47.07 71 1.064 3480K 1633 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.3 3.7 4.6 8.2 3.6 0.0 0.0 129.7 128.0 65.78 41.30 43 1.530 3480K 1633 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 193.2 9.78 5.76 28 0.349 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.9 total, 12.8 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 10.07 GB write, 798.28 MB/s write, 8.33 GB read, 660.89 MB/s read, 75.6 seconds Interval compaction: 10.07 GB write, 807.90 MB/s write, 8.33 GB read, 668.85 MB/s read, 75.6 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 18, total-stops: 18, interval: 36 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 686462 Average: 16.8512 StdDev: 121.93 Min: 0 Median: 1.7600 Max: 8646 Percentiles: P50: 1.76 P75: 2.71 P99: 250.43 P99.9: 1288.14 P99.99: 2795.12 ------------------------------------------------------ [ 0, 1 ] 85060 12.391% 12.391% ## ( 1, 2 ] 339711 49.487% 61.878% ########## ( 2, 3 ] 127092 18.514% 80.392% #### ( 3, 4 ] 43564 6.346% 86.739% # ( 4, 6 ] 40094 5.841% 92.579% # ( 6, 10 ] 11065 1.612% 94.191% ( 10, 15 ] 2297 0.335% 94.526% ( 15, 22 ] 326 0.047% 94.573% ( 22, 34 ] 40 0.006% 94.579% ( 34, 51 ] 17 0.002% 94.581% ( 51, 76 ] 38 0.006% 94.587% ( 76, 110 ] 4971 0.724% 95.311% ( 110, 170 ] 18295 2.665% 97.976% # ( 170, 250 ] 7019 1.022% 98.999% ( 250, 380 ] 2528 0.368% 99.367% ( 380, 580 ] 2279 0.332% 99.699% ( 580, 870 ] 799 0.116% 99.815% ( 870, 1300 ] 597 0.087% 99.902% ( 1300, 1900 ] 391 0.057% 99.959% ( 1900, 2900 ] 235 0.034% 99.994% ( 2900, 4400 ] 240 0.035% 100.029% ( 4400, 6600 ] 66 0.010% 100.038% ( 6600, 9900 ] 14 0.002% 100.040% ** Level 1 read latency histogram (micros): Count: 972 Average: 361.2891 StdDev: 930.11 Min: 80 Median: 156.8230 Max: 12944 Percentiles: P50: 156.82 P75: 238.71 P99: 5104.00 P99.9: 10014.80 P99.99: 12944.00 ------------------------------------------------------ ( 76, 110 ] 120 12.346% 12.346% ## ( 110, 170 ] 469 48.251% 60.597% ########## ( 170, 250 ] 163 16.770% 77.366% ### ( 250, 380 ] 76 7.819% 85.185% ## ( 380, 580 ] 53 5.453% 90.638% # ( 580, 870 ] 32 3.292% 93.930% # ( 870, 1300 ] 14 1.440% 95.370% ( 1300, 1900 ] 19 1.955% 97.325% ( 1900, 2900 ] 10 1.029% 98.354% ( 2900, 4400 ] 5 0.514% 98.868% ( 4400, 6600 ] 4 0.412% 99.280% ( 6600, 9900 ] 6 0.617% 99.897% ( 9900, 14000 ] 1 0.103% 100.000% ** Level 2 read latency histogram (micros): Count: 65435 Average: 9.1620 StdDev: 129.53 Min: 1 Median: 2.0190 Max: 6791 Percentiles: P50: 2.02 P75: 2.93 P99: 10.26 P99.9: 2218.83 P99.99: 5661.87 ------------------------------------------------------ [ 0, 1 ] 1560 2.384% 2.384% ( 1, 2 ] 30815 47.093% 49.477% ######### ( 2, 3 ] 18054 27.591% 77.067% ###### ( 3, 4 ] 6194 9.466% 86.533% ## ( 4, 6 ] 6268 9.579% 96.112% ## ( 6, 10 ] 1872 2.861% 98.973% # ( 10, 15 ] 345 0.527% 99.500% ( 15, 22 ] 34 0.052% 99.552% ( 22, 34 ] 12 0.018% 99.571% ( 34, 51 ] 15 0.023% 99.593% ( 51, 76 ] 12 0.018% 99.612% ( 76, 110 ] 7 0.011% 99.623% ( 110, 170 ] 7 0.011% 99.633% ( 170, 250 ] 12 0.018% 99.652% ( 250, 380 ] 14 0.021% 99.673% ( 380, 580 ] 20 0.031% 99.704% ( 580, 870 ] 37 0.057% 99.760% ( 870, 1300 ] 50 0.076% 99.836% ( 1300, 1900 ] 32 0.049% 99.885% ( 1900, 2900 ] 30 0.046% 99.931% ( 2900, 4400 ] 31 0.047% 99.979% ( 4400, 6600 ] 13 0.020% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 781520 Average: 11.1983 StdDev: 181.26 Min: 1 Median: 2.2746 Max: 49405 Percentiles: P50: 2.27 P75: 3.23 P99: 9.97 P99.9: 248.16 P99.99: 659.81 ------------------------------------------------------ [ 0, 1 ] 10685 1.367% 1.367% ( 1, 2 ] 314044 40.184% 41.551% ######## ( 2, 3 ] 240428 30.764% 72.315% ###### ( 3, 4 ] 90249 11.548% 83.863% ## ( 4, 6 ] 95084 12.167% 96.030% ## ( 6, 10 ] 23418 2.996% 99.026% # ( 10, 15 ] 4783 0.612% 99.638% ( 15, 22 ] 656 0.084% 99.722% ( 22, 34 ] 88 0.011% 99.733% ( 34, 51 ] 76 0.010% 99.743% ( 51, 76 ] 80 0.010% 99.753% ( 76, 110 ] 243 0.031% 99.784% ( 110, 170 ] 628 0.080% 99.865% ( 170, 250 ] 283 0.036% 99.901% ( 250, 380 ] 211 0.027% 99.928% ( 380, 580 ] 351 0.045% 99.973% ( 580, 870 ] 490 0.063% 100.035% ( 870, 1300 ] 579 0.074% 100.110% ( 1300, 1900 ] 478 0.061% 100.171% ( 1900, 2900 ] 442 0.057% 100.227% ( 2900, 4400 ] 459 0.059% 100.286% ( 4400, 6600 ] 157 0.020% 100.306% ( 6600, 9900 ] 44 0.006% 100.312% ( 9900, 14000 ] 9 0.001% 100.313% ( 14000, 22000 ] 8 0.001% 100.314% ( 33000, 50000 ] 3 0.000% 100.314% ** Level 4 read latency histogram (micros): Count: 586442 Average: 21.7306 StdDev: 283.47 Min: 1 Median: 2.2920 Max: 55105 Percentiles: P50: 2.29 P75: 3.33 P99: 209.86 P99.9: 1679.83 P99.99: 2273.46 ------------------------------------------------------ [ 0, 1 ] 8103 1.382% 1.382% ( 1, 2 ] 233266 39.776% 41.158% ######## ( 2, 3 ] 177560 30.278% 71.436% ###### ( 3, 4 ] 64055 10.923% 82.358% ## ( 4, 6 ] 68072 11.608% 93.966% ## ( 6, 10 ] 17607 3.002% 96.968% # ( 10, 15 ] 3471 0.592% 97.560% ( 15, 22 ] 504 0.086% 97.646% ( 22, 34 ] 57 0.010% 97.656% ( 34, 51 ] 34 0.006% 97.662% ( 51, 76 ] 74 0.013% 97.674% ( 76, 110 ] 1450 0.247% 97.922% ( 110, 170 ] 5345 0.911% 98.833% ( 170, 250 ] 1968 0.336% 99.169% ( 250, 380 ] 995 0.170% 99.338% ( 380, 580 ] 964 0.164% 99.503% ( 580, 870 ] 797 0.136% 99.638% ( 870, 1300 ] 999 0.170% 99.809% ( 1300, 1900 ] 846 0.144% 99.953% ( 1900, 2900 ] 582 0.099% 100.052% ( 2900, 4400 ] 536 0.091% 100.144% ( 4400, 6600 ] 240 0.041% 100.185% ( 6600, 9900 ] 75 0.013% 100.197% ( 9900, 14000 ] 15 0.003% 100.200% ( 14000, 22000 ] 17 0.003% 100.203% ( 22000, 33000 ] 6 0.001% 100.204% ( 33000, 50000 ] 5 0.001% 100.205% ( 50000, 75000 ] 1 0.000% 100.205% ** Level 5 read latency histogram (micros): Count: 83398 Average: 407.1969 StdDev: 1092.27 Min: 74 Median: 158.0619 Max: 71975 Percentiles: P50: 158.06 P75: 245.77 P99: 4871.68 P99.9: 8614.59 P99.99: 10080.45 ------------------------------------------------------ ( 51, 76 ] 3 0.004% 0.004% ( 76, 110 ] 10471 12.555% 12.559% ### ( 110, 170 ] 38981 46.741% 59.300% ######### ( 170, 250 ] 13825 16.577% 75.877% ### ( 250, 380 ] 5702 6.837% 82.714% # ( 380, 580 ] 5271 6.320% 89.035% # ( 580, 870 ] 2279 2.733% 91.767% # ( 870, 1300 ] 1788 2.144% 93.911% ( 1300, 1900 ] 1429 1.713% 95.625% ( 1900, 2900 ] 1224 1.468% 97.092% ( 2900, 4400 ] 1418 1.700% 98.793% ( 4400, 6600 ] 807 0.968% 99.760% ( 6600, 9900 ] 191 0.229% 99.989% ( 9900, 14000 ] 15 0.018% 100.007% ( 14000, 22000 ] 3 0.004% 100.011% ( 22000, 33000 ] 3 0.004% 100.014% ( 33000, 50000 ] 10 0.012% 100.026% ( 50000, 75000 ] 7 0.008% 100.035% ** DB Stats ** Uptime(secs): 12.9 total, 4.6 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 152.45 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.119 H:M:S, 31.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 211.97 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.744 H:M:S, 16.0 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/27-07:17:17 ... thread 0: (1000000,3000000) ops and (108268.6,136413.2) ops/second in (9.236289,21.992004) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 19/12 4.35 GB 263.3 3.8 0.0 3.8 6.6 2.8 0.0 2.3 133.9 232.9 29.15 23.77 54 0.540 3964K 13K 0.0 0.0 L1 9/9 1.02 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 20/0 991.13 MB 1.0 1.3 0.6 0.6 1.3 0.6 0.0 2.0 114.2 113.2 11.36 6.44 5 2.271 392K 0 0.0 0.0 L4 174/0 9.76 GB 1.0 5.5 3.1 2.4 5.4 3.0 0.0 1.7 119.2 117.0 47.49 28.64 32 1.484 1489K 0 0.0 0.0 L5 1132/0 93.01 GB 1.0 0.0 0.0 0.0 0.0 0.0 3.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1354/21 109.12 GB 0.0 10.6 3.7 6.9 13.3 6.4 3.0 4.7 123.4 154.9 88.00 58.85 91 0.967 5845K 13K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.6 3.7 6.9 13.3 6.4 3.0 4.7 123.4 154.9 88.00 58.85 91 0.967 5845K 13K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.6 3.7 6.9 10.5 3.6 0.0 0.0 145.1 143.4 74.84 50.16 47 1.592 5845K 13K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 220.2 13.16 8.69 44 0.299 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.1 total, 22.0 interval Flush(GB): cumulative 2.830, interval 2.830 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.31 GB write, 615.54 MB/s write, 10.61 GB read, 490.45 MB/s read, 88.0 seconds Interval compaction: 13.31 GB write, 619.85 MB/s write, 10.61 GB read, 493.88 MB/s read, 88.0 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 12, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 12, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 18, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 26, total-stops: 18, interval: 44 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1402782 Average: 17.6608 StdDev: 95.04 Min: 0 Median: 1.6033 Max: 8646 Percentiles: P50: 1.60 P75: 2.34 P99: 244.70 P99.9: 712.56 P99.99: 1331.40 ------------------------------------------------------ [ 0, 1 ] 262340 18.701% 18.701% #### ( 1, 2 ] 727719 51.877% 70.578% ########## ( 2, 3 ] 179998 12.832% 83.410% ### ( 3, 4 ] 59180 4.219% 87.629% # ( 4, 6 ] 48198 3.436% 91.064% # ( 6, 10 ] 12908 0.920% 91.985% ( 10, 15 ] 2713 0.193% 92.178% ( 15, 22 ] 399 0.028% 92.206% ( 22, 34 ] 63 0.004% 92.211% ( 34, 51 ] 20 0.001% 92.212% ( 51, 76 ] 60 0.004% 92.217% ( 76, 110 ] 16932 1.207% 93.424% ( 110, 170 ] 60650 4.324% 97.747% # ( 170, 250 ] 18821 1.342% 99.089% ( 250, 380 ] 5816 0.415% 99.503% ( 380, 580 ] 5037 0.359% 99.863% ( 580, 870 ] 1149 0.082% 99.944% ( 870, 1300 ] 618 0.044% 99.989% ( 1300, 1900 ] 396 0.028% 100.017% ( 1900, 2900 ] 252 0.018% 100.035% ( 2900, 4400 ] 247 0.018% 100.052% ( 4400, 6600 ] 67 0.005% 100.057% ( 6600, 9900 ] 14 0.001% 100.058% ** Level 1 read latency histogram (micros): Count: 2585 Average: 238.3903 StdDev: 582.12 Min: 80 Median: 148.0587 Max: 12944 Percentiles: P50: 148.06 P75: 196.68 P99: 1915.00 P99.9: 9028.25 P99.99: 12940.15 ------------------------------------------------------ ( 76, 110 ] 407 15.745% 15.745% ### ( 110, 170 ] 1396 54.004% 69.749% ########### ( 170, 250 ] 407 15.745% 85.493% ### ( 250, 380 ] 158 6.112% 91.605% # ( 380, 580 ] 117 4.526% 96.132% # ( 580, 870 ] 41 1.586% 97.718% ( 870, 1300 ] 14 0.542% 98.259% ( 1300, 1900 ] 19 0.735% 98.994% ( 1900, 2900 ] 10 0.387% 99.381% ( 2900, 4400 ] 5 0.193% 99.574% ( 4400, 6600 ] 4 0.155% 99.729% ( 6600, 9900 ] 6 0.232% 99.961% ( 9900, 14000 ] 1 0.039% 100.000% ** Level 2 read latency histogram (micros): Count: 65435 Average: 9.1620 StdDev: 129.53 Min: 1 Median: 2.0190 Max: 6791 Percentiles: P50: 2.02 P75: 2.93 P99: 10.26 P99.9: 2218.83 P99.99: 5661.87 ------------------------------------------------------ [ 0, 1 ] 1560 2.384% 2.384% ( 1, 2 ] 30815 47.093% 49.477% ######### ( 2, 3 ] 18054 27.591% 77.067% ###### ( 3, 4 ] 6194 9.466% 86.533% ## ( 4, 6 ] 6268 9.579% 96.112% ## ( 6, 10 ] 1872 2.861% 98.973% # ( 10, 15 ] 345 0.527% 99.500% ( 15, 22 ] 34 0.052% 99.552% ( 22, 34 ] 12 0.018% 99.571% ( 34, 51 ] 15 0.023% 99.593% ( 51, 76 ] 12 0.018% 99.612% ( 76, 110 ] 7 0.011% 99.623% ( 110, 170 ] 7 0.011% 99.633% ( 170, 250 ] 12 0.018% 99.652% ( 250, 380 ] 14 0.021% 99.673% ( 380, 580 ] 20 0.031% 99.704% ( 580, 870 ] 37 0.057% 99.760% ( 870, 1300 ] 50 0.076% 99.836% ( 1300, 1900 ] 32 0.049% 99.885% ( 1900, 2900 ] 30 0.046% 99.931% ( 2900, 4400 ] 31 0.047% 99.979% ( 4400, 6600 ] 13 0.020% 99.998% ( 6600, 9900 ] 1 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 783720 Average: 11.6011 StdDev: 181.23 Min: 1 Median: 2.2791 Max: 49405 Percentiles: P50: 2.28 P75: 3.25 P99: 11.93 P99.9: 320.84 P99.99: 663.39 ------------------------------------------------------ [ 0, 1 ] 10685 1.363% 1.363% ( 1, 2 ] 314059 40.073% 41.436% ######## ( 2, 3 ] 240472 30.683% 72.120% ###### ( 3, 4 ] 90276 11.519% 83.639% ## ( 4, 6 ] 95117 12.137% 95.775% ## ( 6, 10 ] 23423 2.989% 98.764% # ( 10, 15 ] 4783 0.610% 99.374% ( 15, 22 ] 656 0.084% 99.458% ( 22, 34 ] 89 0.011% 99.469% ( 34, 51 ] 76 0.010% 99.479% ( 51, 76 ] 80 0.010% 99.489% ( 76, 110 ] 563 0.072% 99.561% ( 110, 170 ] 1860 0.237% 99.798% ( 170, 250 ] 624 0.080% 99.878% ( 250, 380 ] 318 0.041% 99.918% ( 380, 580 ] 418 0.053% 99.972% ( 580, 870 ] 496 0.063% 100.035% ( 870, 1300 ] 581 0.074% 100.109% ( 1300, 1900 ] 478 0.061% 100.170% ( 1900, 2900 ] 442 0.056% 100.227% ( 2900, 4400 ] 459 0.059% 100.285% ( 4400, 6600 ] 157 0.020% 100.305% ( 6600, 9900 ] 44 0.006% 100.311% ( 9900, 14000 ] 9 0.001% 100.312% ( 14000, 22000 ] 8 0.001% 100.313% ( 33000, 50000 ] 3 0.000% 100.313% ** Level 4 read latency histogram (micros): Count: 607269 Average: 26.2037 StdDev: 280.18 Min: 1 Median: 2.3488 Max: 55105 Percentiles: P50: 2.35 P75: 3.56 P99: 286.30 P99.9: 1667.73 P99.99: 2273.12 ------------------------------------------------------ [ 0, 1 ] 8103 1.334% 1.334% ( 1, 2 ] 233414 38.437% 39.771% ######## ( 2, 3 ] 178068 29.323% 69.094% ###### ( 3, 4 ] 64426 10.609% 79.703% ## ( 4, 6 ] 68602 11.297% 91.000% ## ( 6, 10 ] 17647 2.906% 93.906% # ( 10, 15 ] 3483 0.574% 94.479% ( 15, 22 ] 511 0.084% 94.563% ( 22, 34 ] 57 0.009% 94.573% ( 34, 51 ] 34 0.006% 94.578% ( 51, 76 ] 77 0.013% 94.591% ( 76, 110 ] 4668 0.769% 95.360% ( 110, 170 ] 16594 2.733% 98.092% # ( 170, 250 ] 4996 0.823% 98.915% ( 250, 380 ] 1849 0.304% 99.219% ( 380, 580 ] 1718 0.283% 99.502% ( 580, 870 ] 890 0.147% 99.649% ( 870, 1300 ] 1005 0.165% 99.814% ( 1300, 1900 ] 848 0.140% 99.954% ( 1900, 2900 ] 585 0.096% 100.050% ( 2900, 4400 ] 537 0.088% 100.139% ( 4400, 6600 ] 240 0.040% 100.178% ( 6600, 9900 ] 75 0.012% 100.191% ( 9900, 14000 ] 15 0.002% 100.193% ( 14000, 22000 ] 17 0.003% 100.196% ( 22000, 33000 ] 6 0.001% 100.197% ( 33000, 50000 ] 5 0.001% 100.198% ( 50000, 75000 ] 1 0.000% 100.198% ** Level 5 read latency histogram (micros): Count: 217259 Average: 257.5598 StdDev: 690.58 Min: 74 Median: 148.7663 Max: 71975 Percentiles: P50: 148.77 P75: 199.01 P99: 3112.91 P99.9: 6411.19 P99.99: 8781.70 ------------------------------------------------------ ( 51, 76 ] 6 0.003% 0.003% ( 76, 110 ] 32531 14.973% 14.976% ### ( 110, 170 ] 117771 54.208% 69.184% ########### ( 170, 250 ] 34841 16.037% 85.220% ### ( 250, 380 ] 11926 5.489% 90.710% # ( 380, 580 ] 10450 4.810% 95.520% # ( 580, 870 ] 2845 1.309% 96.829% ( 870, 1300 ] 1832 0.843% 97.672% ( 1300, 1900 ] 1443 0.664% 98.337% ( 1900, 2900 ] 1240 0.571% 98.907% ( 2900, 4400 ] 1419 0.653% 99.560% ( 4400, 6600 ] 807 0.371% 99.932% ( 6600, 9900 ] 191 0.088% 100.020% ( 9900, 14000 ] 15 0.007% 100.027% ( 14000, 22000 ] 3 0.001% 100.028% ( 22000, 33000 ] 3 0.001% 100.029% ( 33000, 50000 ] 10 0.005% 100.034% ( 50000, 75000 ] 7 0.003% 100.037% ** DB Stats ** Uptime(secs): 22.1 total, 9.2 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 133.31 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:9.181 H:M:S, 41.5 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 106.56 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:5.062 H:M:S, 54.8 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 219.314 micros/op 18221 ops/sec 60.148 seconds 1095996 operations; 17.7 MB/s (274999 of 274999 found) Microseconds per read: Count: 1095996 Average: 219.3159 StdDev: 823.63 Min: 2 Median: 167.0657 Max: 236546 Percentiles: P50: 167.07 P75: 225.19 P99: 729.53 P99.9: 10248.65 P99.99: 31647.49 ------------------------------------------------------ ( 1, 2 ] 6 0.001% 0.001% ( 2, 3 ] 25 0.002% 0.003% ( 3, 4 ] 80 0.007% 0.010% ( 4, 6 ] 108 0.010% 0.020% ( 6, 10 ] 252 0.023% 0.043% ( 10, 15 ] 4560 0.416% 0.459% ( 15, 22 ] 4359 0.398% 0.857% ( 22, 34 ] 10629 0.970% 1.827% ( 34, 51 ] 2665 0.243% 2.070% ( 51, 76 ] 405 0.037% 2.107% ( 76, 110 ] 26867 2.451% 4.558% ( 110, 170 ] 523651 47.779% 52.337% ########## ( 170, 250 ] 360029 32.849% 85.186% ####### ( 250, 380 ] 105863 9.659% 94.845% ## ( 380, 580 ] 40120 3.661% 98.506% # ( 580, 870 ] 10506 0.959% 99.464% ( 870, 1300 ] 2490 0.227% 99.692% ( 1300, 1900 ] 846 0.077% 99.769% ( 1900, 2900 ] 574 0.052% 99.821% ( 2900, 4400 ] 428 0.039% 99.860% ( 4400, 6600 ] 280 0.026% 99.886% ( 6600, 9900 ] 136 0.012% 99.898% ( 9900, 14000 ] 247 0.023% 99.921% ( 14000, 22000 ] 535 0.049% 99.969% ( 22000, 33000 ] 257 0.023% 99.993% ( 33000, 50000 ] 58 0.005% 99.998% ( 50000, 75000 ] 15 0.001% 100.000% ( 75000, 110000 ] 1 0.000% 100.000% ( 170000, 250000 ] 4 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 15385743 rocksdb.bloom.filter.full.positive COUNT : 1244888 rocksdb.bloom.filter.full.true.positive COUNT : 1095530 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 466 rocksdb.memtable.miss COUNT : 1095530 rocksdb.l0.hit COUNT : 43463 rocksdb.l1.hit COUNT : 16358 rocksdb.l2andup.hit COUNT : 1035709 rocksdb.compaction.key.drop.new COUNT : 178089 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3365342 rocksdb.number.keys.read COUNT : 1095996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3473032944 rocksdb.bytes.read COUNT : 1095996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1532 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 45115608 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 16741149544 rocksdb.compact.write.bytes COUNT : 16551472424 rocksdb.flush.write.bytes COUNT : 3435105264 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 34 rocksdb.number.superversion_acquires COUNT : 644 rocksdb.number.superversion_releases COUNT : 614 rocksdb.number.superversion_cleanups COUNT : 163 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 175 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3407262720 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1200128 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21029275559 rocksdb.non.last.level.read.count COUNT : 4936180 rocksdb.block.checksum.compute.count COUNT : 5308734 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 166.249285 P95 : 386.372803 P99 : 728.250488 P100 : 236544.000000 COUNT : 1095996 SUM : 239464792 rocksdb.db.write.micros P50 : 1.696331 P95 : 5.740249 P99 : 25.226364 P100 : 766095.000000 COUNT : 3365342 SUM : 55217541 rocksdb.compaction.times.micros P50 : 1484375.000000 P95 : 3530000.000000 P99 : 40840000.000000 P100 : 44973248.000000 COUNT : 51 SUM : 125776620 rocksdb.compaction.times.cpu_micros P50 : 950000.000000 P95 : 3530000.000000 P99 : 17920000.000000 P100 : 19752627.000000 COUNT : 51 SUM : 75786092 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 773.333333 P95 : 9075.000000 P99 : 12975.000000 P100 : 13605.000000 COUNT : 50 SUM : 88249 rocksdb.compaction.outfile.sync.micros P50 : 6261.538462 P95 : 46600.000000 P99 : 98100.000000 P100 : 154395.000000 COUNT : 168 SUM : 1963499 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 207.500000 P95 : 5261.666667 P99 : 9471.000000 P100 : 11486.000000 COUNT : 113 SUM : 114133 rocksdb.table.open.io.micros P50 : 10696.402878 P95 : 22060.273973 P99 : 31294.246575 P100 : 188026.000000 COUNT : 1532 SUM : 17305720 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.313688 P95 : 104.702426 P99 : 211.150639 P100 : 55109.000000 COUNT : 4052915 SUM : 74265061 rocksdb.read.block.get.micros P50 : 139.443373 P95 : 264.025382 P99 : 573.458717 P100 : 72063.000000 COUNT : 1254299 SUM : 201139668 rocksdb.write.raw.block.micros P50 : 0.573686 P95 : 1.755907 P99 : 4.225652 P100 : 124955.000000 COUNT : 4840843 SUM : 14942730 rocksdb.numfiles.in.singlecompaction P50 : 2.961538 P95 : 5.850000 P99 : 26.440000 P100 : 30.000000 COUNT : 63 SUM : 243 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.572204 P95 : 1931.354060 P99 : 2712.794767 P100 : 765879.000000 COUNT : 229978 SUM : 45135194 rocksdb.sst.read.micros P50 : 2.568603 P95 : 169.904234 P99 : 391.018265 P100 : 71975.000000 COUNT : 4936254 SUM : 256945126 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1095996 SUM : 1095996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3365342 SUM : 3473032944 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 220000.000000 P95 : 916666.666667 P99 : 1137843.000000 P100 : 1137843.000000 COUNT : 50 SUM : 14492005 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 178 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0