[2024-11-27 07:07:41.336597] Starting SPDK v25.01-pre git sha1 345c51d49 / DPDK 24.03.0 initialization... [2024-11-27 07:07:41.336916] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=lib.power:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid3418160 ] Set seed to 1732687661329378 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-27 07:07:41.491178] app.c: 919:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-27 07:07:41.596184] app.c: 612:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-27 07:07:41.596239] app.c: 613:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 3418160' to capture a snapshot of events at runtime. [2024-11-27 07:07:41.596253] app.c: 618:app_setup_trace: *NOTICE*: 'spdk_trace' without parameters will also work if this is the only [2024-11-27 07:07:41.596267] app.c: 619:app_setup_trace: *NOTICE*: SPDK application currently running. [2024-11-27 07:07:41.596281] app.c: 620:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid3418160 for offline analysis/debug. [2024-11-27 07:07:41.597600] reactor.c:1005:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 27 07:07:44 2024 CPU: 72 * Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz CPUCache: 25344 KB 2024/11/27-07:07:51 ... thread 0: (1000000,1000000) ops and (151923.9,151923.9) ops/second in (6.582241,6.582241) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 12/9 1.26 GB 50.1 0.3 0.0 0.3 1.1 0.8 0.0 1.4 39.6 142.3 7.94 3.96 11 0.722 319K 1857 0.0 0.0 L1 2/2 102.15 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 1/1 128.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 30/9 1.66 GB 1.2 2.0 1.6 0.4 2.0 1.6 0.0 1.2 103.0 101.3 20.31 11.10 10 2.031 1695K 5051 0.0 0.0 L4 161/28 11.71 GB 1.0 2.7 1.7 1.0 2.6 1.6 0.0 1.5 123.0 119.1 22.46 13.88 13 1.728 751K 436 0.0 0.0 L5 120/0 9.35 GB 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 326/49 24.20 GB 0.0 5.0 3.3 1.7 5.7 4.0 0.0 7.2 102.0 115.6 50.71 28.94 34 1.491 2766K 7344 0.0 0.0 Int 0/0 0.00 KB 0.0 5.0 3.3 1.7 5.7 4.0 0.0 7.2 102.0 115.6 50.71 28.94 34 1.491 2766K 7344 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 5.0 3.3 1.7 4.9 3.2 0.0 0.0 115.3 112.5 44.85 26.49 24 1.869 2766K 7344 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.8 0.8 0.0 0.0 0.0 139.4 5.86 2.45 10 0.586 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 6.7 total, 6.6 interval Flush(GB): cumulative 0.798, interval 0.798 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 5.72 GB write, 880.42 MB/s write, 5.05 GB read, 776.55 MB/s read, 50.7 seconds Interval compaction: 5.72 GB write, 890.31 MB/s write, 5.05 GB read, 785.27 MB/s read, 50.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 15, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 0, total-stops: 15, interval: 15 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 122232 Average: 75.9919 StdDev: 649.82 Min: 1 Median: 2.2151 Max: 63338 Percentiles: P50: 2.22 P75: 3.47 P99: 1623.05 P99.9: 5488.74 P99.99: 23424.13 ------------------------------------------------------ [ 0, 1 ] 2562 2.096% 2.096% ( 1, 2 ] 51702 42.298% 44.394% ######## ( 2, 3 ] 31854 26.060% 70.455% ##### ( 3, 4 ] 11726 9.593% 80.048% ## ( 4, 6 ] 12672 10.367% 90.415% ## ( 6, 10 ] 3100 2.536% 92.951% # ( 10, 15 ] 744 0.609% 93.560% ( 15, 22 ] 106 0.087% 93.647% ( 22, 34 ] 21 0.017% 93.664% ( 34, 51 ] 16 0.013% 93.677% ( 51, 76 ] 10 0.008% 93.685% ( 76, 110 ] 2 0.002% 93.687% ( 110, 170 ] 48 0.039% 93.726% ( 170, 250 ] 242 0.198% 93.924% ( 250, 380 ] 781 0.639% 94.563% ( 380, 580 ] 2022 1.654% 96.217% ( 580, 870 ] 2244 1.836% 98.053% ( 870, 1300 ] 994 0.813% 98.866% ( 1300, 1900 ] 304 0.249% 99.115% ( 1900, 2900 ] 369 0.302% 99.417% ( 2900, 4400 ] 520 0.425% 99.842% ( 4400, 6600 ] 143 0.117% 99.959% ( 6600, 9900 ] 25 0.020% 99.980% ( 9900, 14000 ] 4 0.003% 99.983% ( 14000, 22000 ] 8 0.007% 99.989% ( 22000, 33000 ] 6 0.005% 99.994% ( 33000, 50000 ] 10 0.008% 100.002% ( 50000, 75000 ] 3 0.002% 100.005% ** Level 1 read latency histogram (micros): Count: 97 Average: 1420.3814 StdDev: 2961.30 Min: 131 Median: 697.1154 Max: 26974 Percentiles: P50: 697.12 P75: 1091.32 P99: 22330.00 P99.9: 26974.00 P99.99: 26974.00 ------------------------------------------------------ ( 110, 170 ] 2 2.062% 2.062% ( 170, 250 ] 3 3.093% 5.155% # ( 250, 380 ] 11 11.340% 16.495% ## ( 380, 580 ] 22 22.680% 39.175% ##### ( 580, 870 ] 26 26.804% 65.979% ##### ( 870, 1300 ] 17 17.526% 83.505% #### ( 1300, 1900 ] 1 1.031% 84.536% ( 1900, 2900 ] 3 3.093% 87.629% # ( 2900, 4400 ] 6 6.186% 93.814% # ( 4400, 6600 ] 4 4.124% 97.938% # ( 6600, 9900 ] 1 1.031% 98.969% ( 22000, 33000 ] 1 1.031% 100.000% ** Level 2 read latency histogram (micros): Count: 333492 Average: 17.3545 StdDev: 373.14 Min: 1 Median: 2.2517 Max: 71627 Percentiles: P50: 2.25 P75: 3.31 P99: 13.77 P99.9: 1819.31 P99.99: 3111.52 ------------------------------------------------------ [ 0, 1 ] 3895 1.168% 1.168% ( 1, 2 ] 138804 41.621% 42.789% ######## ( 2, 3 ] 95520 28.642% 71.432% ###### ( 3, 4 ] 38083 11.419% 82.851% ## ( 4, 6 ] 41664 12.493% 95.344% ## ( 6, 10 ] 10153 3.044% 98.389% # ( 10, 15 ] 2706 0.811% 99.200% ( 15, 22 ] 444 0.133% 99.333% ( 22, 34 ] 60 0.018% 99.351% ( 34, 51 ] 73 0.022% 99.373% ( 51, 76 ] 72 0.022% 99.395% ( 76, 110 ] 17 0.005% 99.400% ( 110, 170 ] 37 0.011% 99.411% ( 170, 250 ] 67 0.020% 99.431% ( 250, 380 ] 252 0.076% 99.507% ( 380, 580 ] 481 0.144% 99.651% ( 580, 870 ] 415 0.124% 99.775% ( 870, 1300 ] 245 0.073% 99.849% ( 1300, 1900 ] 197 0.059% 99.908% ( 1900, 2900 ] 236 0.071% 99.979% ( 2900, 4400 ] 267 0.080% 100.059% ( 4400, 6600 ] 136 0.041% 100.100% ( 6600, 9900 ] 23 0.007% 100.106% ( 9900, 14000 ] 16 0.005% 100.111% ( 14000, 22000 ] 12 0.004% 100.115% ( 22000, 33000 ] 15 0.004% 100.119% ( 33000, 50000 ] 4 0.001% 100.121% ( 50000, 75000 ] 3 0.001% 100.121% ** Level 3 read latency histogram (micros): Count: 526411 Average: 17.0747 StdDev: 357.16 Min: 1 Median: 2.3320 Max: 87644 Percentiles: P50: 2.33 P75: 3.37 P99: 14.08 P99.9: 1238.46 P99.99: 2274.58 ------------------------------------------------------ [ 0, 1 ] 4737 0.900% 0.900% ( 1, 2 ] 204287 38.808% 39.707% ######## ( 2, 3 ] 163207 31.004% 70.711% ###### ( 3, 4 ] 61543 11.691% 82.402% ## ( 4, 6 ] 68596 13.031% 95.433% ### ( 6, 10 ] 16008 3.041% 98.474% # ( 10, 15 ] 3396 0.645% 99.119% ( 15, 22 ] 488 0.093% 99.212% ( 22, 34 ] 124 0.024% 99.235% ( 34, 51 ] 166 0.032% 99.267% ( 51, 76 ] 105 0.020% 99.287% ( 76, 110 ] 40 0.008% 99.294% ( 110, 170 ] 47 0.009% 99.303% ( 170, 250 ] 136 0.026% 99.329% ( 250, 380 ] 436 0.083% 99.412% ( 380, 580 ] 1034 0.196% 99.608% ( 580, 870 ] 1101 0.209% 99.818% ( 870, 1300 ] 506 0.096% 99.914% ( 1300, 1900 ] 274 0.052% 99.966% ( 1900, 2900 ] 340 0.065% 100.030% ( 2900, 4400 ] 394 0.075% 100.105% ( 4400, 6600 ] 124 0.024% 100.129% ( 6600, 9900 ] 29 0.006% 100.134% ( 9900, 14000 ] 15 0.003% 100.137% ( 14000, 22000 ] 27 0.005% 100.142% ( 22000, 33000 ] 27 0.005% 100.147% ( 33000, 50000 ] 9 0.002% 100.149% ( 75000, 110000 ] 1 0.000% 100.149% ** Level 4 read latency histogram (micros): Count: 628766 Average: 27.0076 StdDev: 352.97 Min: 1 Median: 2.3660 Max: 63338 Percentiles: P50: 2.37 P75: 3.47 P99: 535.46 P99.9: 1517.08 P99.99: 2128.89 ------------------------------------------------------ [ 0, 1 ] 5189 0.825% 0.825% ( 1, 2 ] 239154 38.035% 38.861% ######## ( 2, 3 ] 191359 30.434% 69.295% ###### ( 3, 4 ] 76083 12.100% 81.395% ## ( 4, 6 ] 82650 13.145% 94.540% ### ( 6, 10 ] 18939 3.012% 97.552% # ( 10, 15 ] 4224 0.672% 98.224% ( 15, 22 ] 573 0.091% 98.315% ( 22, 34 ] 128 0.020% 98.335% ( 34, 51 ] 196 0.031% 98.366% ( 51, 76 ] 133 0.021% 98.388% ( 76, 110 ] 55 0.009% 98.396% ( 110, 170 ] 100 0.016% 98.412% ( 170, 250 ] 349 0.056% 98.468% ( 250, 380 ] 1155 0.184% 98.651% ( 380, 580 ] 2823 0.449% 99.100% ( 580, 870 ] 3272 0.520% 99.621% ( 870, 1300 ] 1535 0.244% 99.865% ( 1300, 1900 ] 617 0.098% 99.963% ( 1900, 2900 ] 752 0.120% 100.083% ( 2900, 4400 ] 999 0.159% 100.242% ( 4400, 6600 ] 225 0.036% 100.277% ( 6600, 9900 ] 53 0.008% 100.286% ( 9900, 14000 ] 15 0.002% 100.288% ( 14000, 22000 ] 19 0.003% 100.291% ( 22000, 33000 ] 15 0.002% 100.294% ( 33000, 50000 ] 11 0.002% 100.295% ( 50000, 75000 ] 3 0.000% 100.296% ** Level 5 read latency histogram (micros): Count: 7506 Average: 1155.4037 StdDev: 2039.25 Min: 106 Median: 694.2137 Max: 61577 Percentiles: P50: 694.21 P75: 1123.91 P99: 6071.12 P99.9: 31608.50 P99.99: 60411.67 ------------------------------------------------------ ( 76, 110 ] 4 0.053% 0.053% ( 110, 170 ] 48 0.639% 0.693% ( 170, 250 ] 215 2.864% 3.557% # ( 250, 380 ] 793 10.565% 14.122% ## ( 380, 580 ] 1849 24.634% 38.756% ##### ( 580, 870 ] 2143 28.550% 67.306% ###### ( 870, 1300 ] 978 13.030% 80.336% ### ( 1300, 1900 ] 334 4.450% 84.786% # ( 1900, 2900 ] 395 5.262% 90.048% # ( 2900, 4400 ] 558 7.434% 97.482% # ( 4400, 6600 ] 150 1.998% 99.480% ( 6600, 9900 ] 22 0.293% 99.774% ( 9900, 14000 ] 1 0.013% 99.787% ( 14000, 22000 ] 5 0.067% 99.853% ( 22000, 33000 ] 4 0.053% 99.907% ( 33000, 50000 ] 5 0.067% 99.973% ( 50000, 75000 ] 3 0.040% 100.013% ** DB Stats ** Uptime(secs): 6.7 total, 6.6 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 147.81 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:1.812 H:M:S, 27.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 149.45 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.812 H:M:S, 27.5 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 10 num-running-flushes: 1 2024/11/27-07:07:55 ... thread 0: (1000000,2000000) ops and (212193.1,177070.6) ops/second in (4.712689,11.294930) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 16/9 2.30 GB 162.7 1.4 0.0 1.4 3.2 1.8 0.0 1.8 75.4 173.4 19.18 11.70 31 0.619 1468K 7880 0.0 0.0 L1 2/2 102.15 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 971.02 MB 1.0 2.3 1.7 0.5 2.2 1.7 0.0 1.3 104.9 103.2 22.20 12.29 11 2.018 1935K 8280 0.0 0.0 L4 144/0 9.66 GB 1.0 7.0 2.4 4.6 6.2 1.6 0.1 2.6 145.1 128.7 49.68 32.09 25 1.987 1492K 436 0.0 0.0 L5 151/0 11.52 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 331/11 24.52 GB 0.0 10.7 4.1 6.6 11.7 5.1 2.3 6.4 120.6 131.9 91.06 56.08 67 1.359 4897K 16K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.7 4.1 6.6 11.7 5.1 2.3 6.4 120.6 131.9 91.06 56.08 67 1.359 4897K 16K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.7 4.1 6.6 9.9 3.3 0.0 0.0 136.3 125.6 80.59 50.56 41 1.966 4897K 16K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 180.2 10.47 5.51 26 0.403 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 11.4 total, 11.3 interval Flush(GB): cumulative 1.842, interval 1.842 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 11.73 GB write, 1055.97 MB/s write, 10.73 GB read, 966.09 MB/s read, 91.1 seconds Interval compaction: 11.73 GB write, 1062.89 MB/s write, 10.73 GB read, 972.41 MB/s read, 91.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 1, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 1, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 1, total-stops: 22, interval: 23 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 456283 Average: 40.7149 StdDev: 380.34 Min: 0 Median: 1.8313 Max: 63338 Percentiles: P50: 1.83 P75: 2.84 P99: 789.85 P99.9: 3389.76 P99.99: 4239.39 ------------------------------------------------------ [ 0, 1 ] 39265 8.605% 8.605% ## ( 1, 2 ] 227216 49.797% 58.403% ########## ( 2, 3 ] 89698 19.658% 78.061% #### ( 3, 4 ] 31488 6.901% 84.962% # ( 4, 6 ] 30081 6.593% 91.555% # ( 6, 10 ] 8187 1.794% 93.349% ( 10, 15 ] 1563 0.343% 93.691% ( 15, 22 ] 191 0.042% 93.733% ( 22, 34 ] 36 0.008% 93.741% ( 34, 51 ] 33 0.007% 93.748% ( 51, 76 ] 20 0.004% 93.753% ( 76, 110 ] 165 0.036% 93.789% ( 110, 170 ] 3477 0.762% 94.551% ( 170, 250 ] 8255 1.809% 96.360% ( 250, 380 ] 4278 0.938% 97.298% ( 380, 580 ] 4805 1.053% 98.351% ( 580, 870 ] 4095 0.897% 99.248% ( 870, 1300 ] 1655 0.363% 99.611% ( 1300, 1900 ] 478 0.105% 99.716% ( 1900, 2900 ] 605 0.133% 99.848% ( 2900, 4400 ] 725 0.159% 100.007% ( 4400, 6600 ] 171 0.037% 100.045% ( 6600, 9900 ] 34 0.007% 100.052% ( 9900, 14000 ] 6 0.001% 100.053% ( 14000, 22000 ] 12 0.003% 100.056% ( 22000, 33000 ] 9 0.002% 100.058% ( 33000, 50000 ] 11 0.002% 100.060% ( 50000, 75000 ] 3 0.001% 100.061% ** Level 1 read latency histogram (micros): Count: 308 Average: 680.0909 StdDev: 1766.40 Min: 104 Median: 304.4186 Max: 26974 Percentiles: P50: 304.42 P75: 611.35 P99: 6006.00 P99.9: 26974.00 P99.99: 26974.00 ------------------------------------------------------ ( 76, 110 ] 2 0.649% 0.649% ( 110, 170 ] 42 13.636% 14.286% ### ( 170, 250 ] 92 29.870% 44.156% ###### ( 250, 380 ] 43 13.961% 58.117% ### ( 380, 580 ] 48 15.584% 73.701% ### ( 580, 870 ] 37 12.013% 85.714% ## ( 870, 1300 ] 24 7.792% 93.506% ## ( 1300, 1900 ] 2 0.649% 94.156% ( 1900, 2900 ] 4 1.299% 95.455% ( 2900, 4400 ] 8 2.597% 98.052% # ( 4400, 6600 ] 4 1.299% 99.351% ( 6600, 9900 ] 1 0.325% 99.675% ( 22000, 33000 ] 1 0.325% 100.000% ** Level 2 read latency histogram (micros): Count: 359120 Average: 16.6239 StdDev: 360.86 Min: 1 Median: 2.2669 Max: 71627 Percentiles: P50: 2.27 P75: 3.33 P99: 13.62 P99.9: 1782.00 P99.99: 3133.82 ------------------------------------------------------ [ 0, 1 ] 4005 1.115% 1.115% ( 1, 2 ] 147877 41.178% 42.293% ######## ( 2, 3 ] 103719 28.881% 71.174% ###### ( 3, 4 ] 41228 11.480% 82.655% ## ( 4, 6 ] 45405 12.643% 95.298% ### ( 6, 10 ] 11180 3.113% 98.411% # ( 10, 15 ] 2918 0.813% 99.224% ( 15, 22 ] 470 0.131% 99.355% ( 22, 34 ] 64 0.018% 99.372% ( 34, 51 ] 77 0.021% 99.394% ( 51, 76 ] 82 0.023% 99.417% ( 76, 110 ] 17 0.005% 99.421% ( 110, 170 ] 39 0.011% 99.432% ( 170, 250 ] 69 0.019% 99.451% ( 250, 380 ] 261 0.073% 99.524% ( 380, 580 ] 496 0.138% 99.662% ( 580, 870 ] 437 0.122% 99.784% ( 870, 1300 ] 253 0.070% 99.854% ( 1300, 1900 ] 204 0.057% 99.911% ( 1900, 2900 ] 241 0.067% 99.978% ( 2900, 4400 ] 270 0.075% 100.053% ( 4400, 6600 ] 138 0.038% 100.092% ( 6600, 9900 ] 24 0.007% 100.099% ( 9900, 14000 ] 17 0.005% 100.103% ( 14000, 22000 ] 12 0.003% 100.107% ( 22000, 33000 ] 15 0.004% 100.111% ( 33000, 50000 ] 4 0.001% 100.112% ( 50000, 75000 ] 3 0.001% 100.113% ** Level 3 read latency histogram (micros): Count: 656509 Average: 16.7517 StdDev: 328.86 Min: 1 Median: 2.3290 Max: 87644 Percentiles: P50: 2.33 P75: 3.38 P99: 130.50 P99.9: 1218.70 P99.99: 2218.84 ------------------------------------------------------ [ 0, 1 ] 6291 0.958% 0.958% ( 1, 2 ] 255860 38.973% 39.931% ######## ( 2, 3 ] 200950 30.609% 70.540% ###### ( 3, 4 ] 76700 11.683% 82.223% ## ( 4, 6 ] 84462 12.865% 95.088% ### ( 6, 10 ] 20179 3.074% 98.162% # ( 10, 15 ] 4199 0.640% 98.802% ( 15, 22 ] 584 0.089% 98.890% ( 22, 34 ] 144 0.022% 98.912% ( 34, 51 ] 208 0.032% 98.944% ( 51, 76 ] 126 0.019% 98.963% ( 76, 110 ] 67 0.010% 98.974% ( 110, 170 ] 509 0.078% 99.051% ( 170, 250 ] 1109 0.169% 99.220% ( 250, 380 ] 924 0.141% 99.361% ( 380, 580 ] 1522 0.232% 99.593% ( 580, 870 ] 1463 0.223% 99.815% ( 870, 1300 ] 685 0.104% 99.920% ( 1300, 1900 ] 337 0.051% 99.971% ( 1900, 2900 ] 390 0.059% 100.030% ( 2900, 4400 ] 442 0.067% 100.098% ( 4400, 6600 ] 130 0.020% 100.118% ( 6600, 9900 ] 30 0.005% 100.122% ( 9900, 14000 ] 15 0.002% 100.124% ( 14000, 22000 ] 30 0.005% 100.129% ( 22000, 33000 ] 27 0.004% 100.133% ( 33000, 50000 ] 10 0.002% 100.135% ( 75000, 110000 ] 1 0.000% 100.135% ** Level 4 read latency histogram (micros): Count: 1209212 Average: 23.5955 StdDev: 289.45 Min: 1 Median: 2.3407 Max: 63338 Percentiles: P50: 2.34 P75: 3.47 P99: 417.50 P99.9: 1083.72 P99.99: 1290.88 ------------------------------------------------------ [ 0, 1 ] 10564 0.874% 0.874% ( 1, 2 ] 472832 39.102% 39.976% ######## ( 2, 3 ] 355722 29.418% 69.394% ###### ( 3, 4 ] 143905 11.901% 81.295% ## ( 4, 6 ] 149927 12.399% 93.693% ## ( 6, 10 ] 36184 2.992% 96.686% # ( 10, 15 ] 7851 0.649% 97.335% ( 15, 22 ] 985 0.081% 97.416% ( 22, 34 ] 198 0.016% 97.433% ( 34, 51 ] 368 0.030% 97.463% ( 51, 76 ] 290 0.024% 97.487% ( 76, 110 ] 260 0.022% 97.509% ( 110, 170 ] 3956 0.327% 97.836% ( 170, 250 ] 8484 0.702% 98.537% ( 250, 380 ] 4537 0.375% 98.913% ( 380, 580 ] 5637 0.466% 99.379% ( 580, 870 ] 5180 0.428% 99.807% ( 870, 1300 ] 2259 0.187% 99.994% ( 1300, 1900 ] 908 0.075% 100.069% ( 1900, 2900 ] 1066 0.088% 100.157% ( 2900, 4400 ] 1310 0.108% 100.266% ( 4400, 6600 ] 280 0.023% 100.289% ( 6600, 9900 ] 66 0.005% 100.294% ( 9900, 14000 ] 22 0.002% 100.296% ( 14000, 22000 ] 29 0.002% 100.298% ( 22000, 33000 ] 18 0.001% 100.300% ( 33000, 50000 ] 14 0.001% 100.301% ( 50000, 75000 ] 3 0.000% 100.301% ** Level 5 read latency histogram (micros): Count: 22477 Average: 632.8980 StdDev: 1347.18 Min: 94 Median: 327.6892 Max: 61577 Percentiles: P50: 327.69 P75: 658.05 P99: 4383.08 P99.9: 9834.41 P99.99: 49298.18 ------------------------------------------------------ ( 76, 110 ] 123 0.547% 0.547% ( 110, 170 ] 2866 12.751% 13.298% ### ( 170, 250 ] 6425 28.585% 41.883% ###### ( 250, 380 ] 3053 13.583% 55.466% ### ( 380, 580 ] 3533 15.718% 71.184% ### ( 580, 870 ] 3187 14.179% 85.363% ### ( 870, 1300 ] 1409 6.269% 91.631% # ( 1300, 1900 ] 454 2.020% 93.651% ( 1900, 2900 ] 521 2.318% 95.969% ( 2900, 4400 ] 689 3.065% 99.035% # ( 4400, 6600 ] 171 0.761% 99.795% ( 6600, 9900 ] 24 0.107% 99.902% ( 9900, 14000 ] 2 0.009% 99.911% ( 14000, 22000 ] 6 0.027% 99.938% ( 22000, 33000 ] 6 0.027% 99.964% ( 33000, 50000 ] 6 0.027% 99.991% ( 50000, 75000 ] 3 0.013% 100.004% ** DB Stats ** Uptime(secs): 11.4 total, 4.7 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 173.11 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:2.488 H:M:S, 21.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 208.84 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.676 H:M:S, 14.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 2024/11/27-07:08:00 ... thread 0: (1000000,3000000) ops and (198257.0,183611.0) ops/second in (5.043958,16.338888) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 3.27 GB 131.4 2.6 0.0 2.6 5.5 2.8 0.0 1.9 99.4 205.3 27.20 19.22 52 0.523 2746K 13K 0.0 0.0 L1 2/2 102.15 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 971.02 MB 1.0 2.3 1.7 0.5 2.2 1.7 0.0 1.3 104.9 103.2 22.20 12.29 11 2.018 1935K 8280 0.0 0.0 L4 144/0 9.66 GB 1.0 7.0 2.4 4.6 6.2 1.6 0.1 2.6 145.1 128.7 49.68 32.09 25 1.987 1492K 436 0.0 0.0 L5 151/0 11.52 GB 0.1 0.0 0.0 0.0 0.0 0.0 2.2 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 332/14 25.50 GB 0.0 12.0 4.1 7.8 13.9 6.1 2.3 4.9 123.6 144.0 99.08 63.60 88 1.126 6174K 22K 0.0 0.0 Int 0/0 0.00 KB 0.0 12.0 4.1 7.8 13.9 6.1 2.3 4.9 123.6 144.0 99.08 63.60 88 1.126 6174K 22K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 12.0 4.1 7.8 11.1 3.3 0.0 0.0 143.4 133.2 85.40 55.24 46 1.857 6174K 22K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 211.5 13.68 8.37 42 0.326 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 16.4 total, 16.3 interval Flush(GB): cumulative 2.825, interval 2.825 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 13.93 GB write, 869.09 MB/s write, 11.96 GB read, 745.88 MB/s read, 99.1 seconds Interval compaction: 13.93 GB write, 873.02 MB/s write, 11.96 GB read, 749.25 MB/s read, 99.1 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 5, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 5, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 22, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 5, total-stops: 22, interval: 27 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 810333 Average: 32.5317 StdDev: 288.57 Min: 0 Median: 1.6762 Max: 63338 Percentiles: P50: 1.68 P75: 2.58 P99: 553.59 P99.9: 2250.36 P99.99: 3593.03 ------------------------------------------------------ [ 0, 1 ] 122601 15.130% 15.130% ### ( 1, 2 ] 417844 51.564% 66.694% ########## ( 2, 3 ] 115788 14.289% 80.983% ### ( 3, 4 ] 41580 5.131% 86.114% # ( 4, 6 ] 35849 4.424% 90.538% # ( 6, 10 ] 9568 1.181% 91.719% ( 10, 15 ] 1856 0.229% 91.948% ( 15, 22 ] 267 0.033% 91.981% ( 22, 34 ] 58 0.007% 91.988% ( 34, 51 ] 41 0.005% 91.993% ( 51, 76 ] 24 0.003% 91.996% ( 76, 110 ] 760 0.094% 92.090% ( 110, 170 ] 15390 1.899% 93.989% ( 170, 250 ] 28663 3.537% 97.526% # ( 250, 380 ] 7082 0.874% 98.400% ( 380, 580 ] 5598 0.691% 99.091% ( 580, 870 ] 4182 0.516% 99.607% ( 870, 1300 ] 1677 0.207% 99.814% ( 1300, 1900 ] 482 0.059% 99.874% ( 1900, 2900 ] 607 0.075% 99.949% ( 2900, 4400 ] 725 0.089% 100.038% ( 4400, 6600 ] 171 0.021% 100.059% ( 6600, 9900 ] 34 0.004% 100.063% ( 9900, 14000 ] 6 0.001% 100.064% ( 14000, 22000 ] 12 0.001% 100.066% ( 22000, 33000 ] 9 0.001% 100.067% ( 33000, 50000 ] 11 0.001% 100.068% ( 50000, 75000 ] 3 0.000% 100.068% ** Level 1 read latency histogram (micros): Count: 684 Average: 414.4503 StdDev: 1210.23 Min: 99 Median: 216.6019 Max: 26974 Percentiles: P50: 216.60 P75: 322.80 P99: 4242.50 P99.9: 25476.00 P99.99: 26974.00 ------------------------------------------------------ ( 76, 110 ] 5 0.731% 0.731% ( 110, 170 ] 157 22.953% 23.684% ##### ( 170, 250 ] 309 45.175% 68.860% ######### ( 250, 380 ] 75 10.965% 79.825% ## ( 380, 580 ] 57 8.333% 88.158% ## ( 580, 870 ] 37 5.409% 93.567% # ( 870, 1300 ] 24 3.509% 97.076% # ( 1300, 1900 ] 2 0.292% 97.368% ( 1900, 2900 ] 4 0.585% 97.953% ( 2900, 4400 ] 8 1.170% 99.123% ( 4400, 6600 ] 4 0.585% 99.708% ( 6600, 9900 ] 1 0.146% 99.854% ( 22000, 33000 ] 1 0.146% 100.000% ** Level 2 read latency histogram (micros): Count: 359120 Average: 16.6239 StdDev: 360.86 Min: 1 Median: 2.2669 Max: 71627 Percentiles: P50: 2.27 P75: 3.33 P99: 13.62 P99.9: 1782.00 P99.99: 3133.82 ------------------------------------------------------ [ 0, 1 ] 4005 1.115% 1.115% ( 1, 2 ] 147877 41.178% 42.293% ######## ( 2, 3 ] 103719 28.881% 71.174% ###### ( 3, 4 ] 41228 11.480% 82.655% ## ( 4, 6 ] 45405 12.643% 95.298% ### ( 6, 10 ] 11180 3.113% 98.411% # ( 10, 15 ] 2918 0.813% 99.224% ( 15, 22 ] 470 0.131% 99.355% ( 22, 34 ] 64 0.018% 99.372% ( 34, 51 ] 77 0.021% 99.394% ( 51, 76 ] 82 0.023% 99.417% ( 76, 110 ] 17 0.005% 99.421% ( 110, 170 ] 39 0.011% 99.432% ( 170, 250 ] 69 0.019% 99.451% ( 250, 380 ] 261 0.073% 99.524% ( 380, 580 ] 496 0.138% 99.662% ( 580, 870 ] 437 0.122% 99.784% ( 870, 1300 ] 253 0.070% 99.854% ( 1300, 1900 ] 204 0.057% 99.911% ( 1900, 2900 ] 241 0.067% 99.978% ( 2900, 4400 ] 270 0.075% 100.053% ( 4400, 6600 ] 138 0.038% 100.092% ( 6600, 9900 ] 24 0.007% 100.099% ( 9900, 14000 ] 17 0.005% 100.103% ( 14000, 22000 ] 12 0.003% 100.107% ( 22000, 33000 ] 15 0.004% 100.111% ( 33000, 50000 ] 4 0.001% 100.112% ( 50000, 75000 ] 3 0.001% 100.113% ** Level 3 read latency histogram (micros): Count: 660671 Average: 17.8763 StdDev: 328.17 Min: 1 Median: 2.3393 Max: 87644 Percentiles: P50: 2.34 P75: 3.42 P99: 224.54 P99.9: 1216.84 P99.99: 2219.52 ------------------------------------------------------ [ 0, 1 ] 6291 0.952% 0.952% ( 1, 2 ] 255860 38.727% 39.680% ######## ( 2, 3 ] 200950 30.416% 70.096% ###### ( 3, 4 ] 76700 11.609% 81.705% ## ( 4, 6 ] 84462 12.784% 94.489% ### ( 6, 10 ] 20179 3.054% 97.544% # ( 10, 15 ] 4199 0.636% 98.179% ( 15, 22 ] 584 0.088% 98.268% ( 22, 34 ] 144 0.022% 98.289% ( 34, 51 ] 208 0.031% 98.321% ( 51, 76 ] 126 0.019% 98.340% ( 76, 110 ] 145 0.022% 98.362% ( 110, 170 ] 1923 0.291% 98.653% ( 170, 250 ] 3364 0.509% 99.162% ( 250, 380 ] 1224 0.185% 99.347% ( 380, 580 ] 1627 0.246% 99.594% ( 580, 870 ] 1471 0.223% 99.816% ( 870, 1300 ] 686 0.104% 99.920% ( 1300, 1900 ] 337 0.051% 99.971% ( 1900, 2900 ] 391 0.059% 100.030% ( 2900, 4400 ] 442 0.067% 100.097% ( 4400, 6600 ] 130 0.020% 100.117% ( 6600, 9900 ] 30 0.005% 100.121% ( 9900, 14000 ] 15 0.002% 100.124% ( 14000, 22000 ] 30 0.005% 100.128% ( 22000, 33000 ] 27 0.004% 100.132% ( 33000, 50000 ] 10 0.002% 100.134% ( 75000, 110000 ] 1 0.000% 100.134% ** Level 4 read latency histogram (micros): Count: 1248281 Average: 28.8288 StdDev: 286.64 Min: 1 Median: 2.3951 Max: 63338 Percentiles: P50: 2.40 P75: 3.67 P99: 430.01 P99.9: 1078.41 P99.99: 1291.13 ------------------------------------------------------ [ 0, 1 ] 10564 0.846% 0.846% ( 1, 2 ] 472905 37.884% 38.731% ######## ( 2, 3 ] 356020 28.521% 67.252% ###### ( 3, 4 ] 144061 11.541% 78.792% ## ( 4, 6 ] 150148 12.028% 90.821% ## ( 6, 10 ] 36210 2.901% 93.722% # ( 10, 15 ] 7871 0.631% 94.352% ( 15, 22 ] 991 0.079% 94.431% ( 22, 34 ] 199 0.016% 94.447% ( 34, 51 ] 368 0.029% 94.477% ( 51, 76 ] 290 0.023% 94.500% ( 76, 110 ] 918 0.074% 94.574% ( 110, 170 ] 16485 1.321% 95.894% ( 170, 250 ] 29868 2.393% 98.287% ( 250, 380 ] 7285 0.584% 98.871% ( 380, 580 ] 6464 0.518% 99.388% ( 580, 870 ] 5286 0.423% 99.812% ( 870, 1300 ] 2271 0.182% 99.994% ( 1300, 1900 ] 912 0.073% 100.067% ( 1900, 2900 ] 1067 0.085% 100.152% ( 2900, 4400 ] 1310 0.105% 100.257% ( 4400, 6600 ] 280 0.022% 100.280% ( 6600, 9900 ] 66 0.005% 100.285% ( 9900, 14000 ] 22 0.002% 100.287% ( 14000, 22000 ] 29 0.002% 100.289% ( 22000, 33000 ] 18 0.001% 100.291% ( 33000, 50000 ] 14 0.001% 100.292% ( 50000, 75000 ] 3 0.000% 100.292% ** Level 5 read latency histogram (micros): Count: 50211 Average: 390.8865 StdDev: 928.55 Min: 91 Median: 216.0920 Max: 61577 Percentiles: P50: 216.09 P75: 332.88 P99: 3779.30 P99.9: 6545.82 P99.99: 41440.22 ------------------------------------------------------ ( 76, 110 ] 602 1.199% 1.199% ( 110, 170 ] 11891 23.682% 24.881% ##### ( 170, 250 ] 21891 43.598% 68.479% ######### ( 250, 380 ] 5136 10.229% 78.708% ## ( 380, 580 ] 4133 8.231% 86.939% ## ( 580, 870 ] 3259 6.491% 93.430% # ( 870, 1300 ] 1417 2.822% 96.252% # ( 1300, 1900 ] 455 0.906% 97.158% ( 1900, 2900 ] 521 1.038% 98.196% ( 2900, 4400 ] 689 1.372% 99.568% ( 4400, 6600 ] 171 0.341% 99.908% ( 6600, 9900 ] 24 0.048% 99.956% ( 9900, 14000 ] 2 0.004% 99.960% ( 14000, 22000 ] 6 0.012% 99.972% ( 22000, 33000 ] 6 0.012% 99.984% ( 33000, 50000 ] 6 0.012% 99.996% ( 50000, 75000 ] 3 0.006% 100.002% ** DB Stats ** Uptime(secs): 16.4 total, 5.0 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 179.87 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.767 H:M:S, 22.9 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 195.12 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:1.279 H:M:S, 25.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 20000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 19378.7 MB (estimated) FileSize: 19378.7 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 389.674 micros/op 10229 ops/sec 20.235 seconds 206996 operations; 9.9 MB/s (51999 of 51999 found) Microseconds per read: Count: 206996 Average: 389.6892 StdDev: 1148.22 Min: 2 Median: 232.9254 Max: 90115 Percentiles: P50: 232.93 P75: 357.70 P99: 3434.21 P99.9: 15333.61 P99.99: 47581.03 ------------------------------------------------------ ( 1, 2 ] 9 0.004% 0.004% ( 2, 3 ] 53 0.026% 0.030% ( 3, 4 ] 79 0.038% 0.068% ( 4, 6 ] 193 0.093% 0.161% ( 6, 10 ] 325 0.157% 0.318% ( 10, 15 ] 1602 0.774% 1.092% ( 15, 22 ] 2877 1.390% 2.482% ( 22, 34 ] 4574 2.210% 4.692% ( 34, 51 ] 1616 0.781% 5.473% ( 51, 76 ] 310 0.150% 5.622% ( 76, 110 ] 99 0.048% 5.670% ( 110, 170 ] 16378 7.912% 13.582% ## ( 170, 250 ] 95838 46.299% 59.882% ######### ( 250, 380 ] 37774 18.249% 78.130% #### ( 380, 580 ] 23339 11.275% 89.406% ## ( 580, 870 ] 11541 5.575% 94.981% # ( 870, 1300 ] 4785 2.312% 97.293% ( 1300, 1900 ] 1693 0.818% 98.111% ( 1900, 2900 ] 1247 0.602% 98.713% ( 2900, 4400 ] 1668 0.806% 99.519% ( 4400, 6600 ] 489 0.236% 99.755% ( 6600, 9900 ] 194 0.094% 99.849% ( 9900, 14000 ] 87 0.042% 99.891% ( 14000, 22000 ] 114 0.055% 99.946% ( 22000, 33000 ] 69 0.033% 99.979% ( 33000, 50000 ] 26 0.013% 99.992% ( 50000, 75000 ] 13 0.006% 99.998% ( 75000, 110000 ] 4 0.002% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 3442095 rocksdb.bloom.filter.full.positive COUNT : 239733 rocksdb.bloom.filter.full.true.positive COUNT : 206124 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 872 rocksdb.memtable.miss COUNT : 206124 rocksdb.l0.hit COUNT : 26963 rocksdb.l1.hit COUNT : 923 rocksdb.l2andup.hit COUNT : 178238 rocksdb.compaction.key.drop.new COUNT : 905569 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3248004 rocksdb.number.keys.read COUNT : 206996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3351940128 rocksdb.bytes.read COUNT : 206996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 501 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 6655529 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 14078354560 rocksdb.compact.write.bytes COUNT : 13143842232 rocksdb.flush.write.bytes COUNT : 3297577155 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 22 rocksdb.number.superversion_acquires COUNT : 617 rocksdb.number.superversion_releases COUNT : 598 rocksdb.number.superversion_cleanups COUNT : 153 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3276194816 rocksdb.memtable.garbage.bytes.at.flush COUNT : 6048768 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 14318915442 rocksdb.non.last.level.read.count COUNT : 3432415 rocksdb.block.checksum.compute.count COUNT : 3654187 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 232.298279 P95 : 871.820201 P99 : 3433.249551 P100 : 90112.000000 COUNT : 206996 SUM : 80479873 rocksdb.db.write.micros P50 : 1.600172 P95 : 5.203812 P99 : 8.278864 P100 : 531256.000000 COUNT : 3248004 SUM : 15831863 rocksdb.compaction.times.micros P50 : 1742500.000000 P95 : 2776315.789474 P99 : 2875263.157895 P100 : 2892948.000000 COUNT : 47 SUM : 86328312 rocksdb.compaction.times.cpu_micros P50 : 1157500.000000 P95 : 1592992.000000 P99 : 1592992.000000 P100 : 1592992.000000 COUNT : 47 SUM : 56152980 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 1061.111111 P95 : 6270.000000 P99 : 9141.000000 P100 : 9841.000000 COUNT : 46 SUM : 83060 rocksdb.compaction.outfile.sync.micros P50 : 2347.368421 P95 : 8563.500000 P99 : 13166.333333 P100 : 34574.000000 COUNT : 161 SUM : 503454 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 208.095238 P95 : 2700.000000 P99 : 4385.000000 P100 : 5196.000000 COUNT : 104 SUM : 47956 rocksdb.table.open.io.micros P50 : 6115.476190 P95 : 18792.000000 P99 : 21998.400000 P100 : 25247.000000 COUNT : 501 SUM : 3365353 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 4.365271 P95 : 9.955679 P99 : 236.856462 P100 : 63346.000000 COUNT : 3411193 SUM : 60139463 rocksdb.read.block.get.micros P50 : 202.939927 P95 : 781.459394 P99 : 2991.735341 P100 : 87648.000000 COUNT : 242723 SUM : 72470313 rocksdb.write.raw.block.micros P50 : 0.537070 P95 : 1.423595 P99 : 4.262872 P100 : 101134.000000 COUNT : 3984039 SUM : 14914770 rocksdb.numfiles.in.singlecompaction P50 : 3.214286 P95 : 5.827273 P99 : 7.000000 P100 : 7.000000 COUNT : 59 SUM : 215 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.559940 P95 : 1129.838013 P99 : 2154.263393 P100 : 531054.000000 COUNT : 34209 SUM : 6658220 rocksdb.sst.read.micros P50 : 2.143816 P95 : 195.837672 P99 : 547.935178 P100 : 87644.000000 COUNT : 3432707 SUM : 118648572 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 206996 SUM : 206996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3248004 SUM : 3351940128 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 231333.333333 P95 : 726600.000000 P99 : 833320.000000 P100 : 845466.000000 COUNT : 46 SUM : 14473709 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 168 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0