[2024-11-27 10:35:34.767127] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-11-27 10:35:34.767276] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1490099 ] Set seed to 1732700134763636 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-11-27 10:35:34.871176] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-11-27 10:35:34.910986] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-11-27 10:35:34.911032] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1490099' to capture a snapshot of events at runtime. [2024-11-27 10:35:34.911044] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1490099 for offline analysis/debug. [2024-11-27 10:35:34.911084] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Wed Nov 27 10:35:39 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/11/27-10:35:43 ... thread 0: (1000000,1000000) ops and (228461.2,228461.2) ops/second in (4.377111,4.377111) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 4.29 GB 88.1 0.7 0.0 0.7 1.6 0.9 0.0 1.9 133.0 288.1 5.68 5.18 17 0.334 767K 724 0.0 0.0 L1 2/2 228.34 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 978.03 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 156/0 9.63 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1112/0 91.54 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1305/14 106.64 GB 0.0 0.7 0.0 0.7 1.6 0.9 0.0 1.9 133.0 288.1 5.68 5.18 17 0.334 767K 724 0.0 0.0 Int 0/0 0.00 KB 0.0 0.7 0.0 0.7 1.6 0.9 0.0 1.9 133.0 288.1 5.68 5.18 17 0.334 767K 724 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 0.7 0.0 0.7 0.7 -0.0 0.0 0.0 263.8 263.5 2.87 2.81 3 0.955 767K 724 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 313.0 2.82 2.37 14 0.201 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 4.5 total, 4.4 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 1.60 GB write, 360.77 MB/s write, 0.74 GB read, 166.59 MB/s read, 5.7 seconds Interval compaction: 1.60 GB write, 373.90 MB/s write, 0.74 GB read, 172.65 MB/s read, 5.7 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 0, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 332810 Average: 12.1832 StdDev: 78.36 Min: 0 Median: 1.4526 Max: 9488 Percentiles: P50: 1.45 P75: 1.90 P99: 224.49 P99.9: 612.10 P99.99: 907.87 ------------------------------------------------------ [ 0, 1 ] 81709 24.551% 24.551% ##### ( 1, 2 ] 187127 56.226% 80.778% ########### ( 2, 3 ] 29826 8.962% 89.739% ## ( 3, 4 ] 8413 2.528% 92.267% # ( 4, 6 ] 5540 1.665% 93.932% ( 6, 10 ] 2905 0.873% 94.805% ( 10, 15 ] 497 0.149% 94.954% ( 15, 22 ] 95 0.029% 94.983% ( 22, 34 ] 46 0.014% 94.997% ( 34, 51 ] 21 0.006% 95.003% ( 51, 76 ] 16 0.005% 95.008% ( 76, 110 ] 2805 0.843% 95.850% ( 110, 170 ] 8459 2.542% 98.392% # ( 170, 250 ] 2970 0.892% 99.285% ( 250, 380 ] 1143 0.343% 99.628% ( 380, 580 ] 869 0.261% 99.889% ( 580, 870 ] 327 0.098% 99.987% ( 870, 1300 ] 99 0.030% 100.017% ( 1300, 1900 ] 42 0.013% 100.030% ( 1900, 2900 ] 25 0.008% 100.037% ( 2900, 4400 ] 15 0.005% 100.042% ( 4400, 6600 ] 11 0.003% 100.045% ( 6600, 9900 ] 5 0.002% 100.047% ** Level 1 read latency histogram (micros): Count: 101 Average: 511.2772 StdDev: 1326.85 Min: 80 Median: 155.3191 Max: 8165 Percentiles: P50: 155.32 P75: 243.33 P99: 8165.00 P99.9: 8165.00 P99.99: 8165.00 ------------------------------------------------------ ( 76, 110 ] 15 14.851% 14.851% ### ( 110, 170 ] 47 46.535% 61.386% ######### ( 170, 250 ] 15 14.851% 76.238% ### ( 250, 380 ] 7 6.931% 83.168% # ( 380, 580 ] 7 6.931% 90.099% # ( 580, 870 ] 2 1.980% 92.079% ( 870, 1300 ] 3 2.970% 95.050% # ( 2900, 4400 ] 1 0.990% 96.040% ( 4400, 6600 ] 2 1.980% 98.020% ( 6600, 9900 ] 2 1.980% 100.000% ** Level 3 read latency histogram (micros): Count: 565 Average: 308.6903 StdDev: 654.90 Min: 80 Median: 158.2955 Max: 8383 Percentiles: P50: 158.30 P75: 241.09 P99: 3575.00 P99.9: 8383.00 P99.99: 8383.00 ------------------------------------------------------ ( 76, 110 ] 70 12.389% 12.389% ## ( 110, 170 ] 264 46.726% 59.115% ######### ( 170, 250 ] 101 17.876% 76.991% #### ( 250, 380 ] 46 8.142% 85.133% ## ( 380, 580 ] 39 6.903% 92.035% # ( 580, 870 ] 20 3.540% 95.575% # ( 870, 1300 ] 9 1.593% 97.168% ( 1300, 1900 ] 7 1.239% 98.407% ( 1900, 2900 ] 2 0.354% 98.761% ( 2900, 4400 ] 3 0.531% 99.292% ( 4400, 6600 ] 2 0.354% 99.646% ( 6600, 9900 ] 2 0.354% 100.000% ** Level 4 read latency histogram (micros): Count: 4557 Average: 281.8589 StdDev: 416.79 Min: 78 Median: 159.0151 Max: 5189 Percentiles: P50: 159.02 P75: 243.35 P99: 2483.71 P99.9: 4369.06 P99.99: 5189.00 ------------------------------------------------------ ( 76, 110 ] 603 13.232% 13.232% ### ( 110, 170 ] 2051 45.008% 58.240% ######### ( 170, 250 ] 833 18.280% 76.520% #### ( 250, 380 ] 377 8.273% 84.793% ## ( 380, 580 ] 301 6.605% 91.398% # ( 580, 870 ] 156 3.423% 94.821% # ( 870, 1300 ] 101 2.216% 97.038% ( 1300, 1900 ] 69 1.514% 98.552% ( 1900, 2900 ] 35 0.768% 99.320% ( 2900, 4400 ] 27 0.592% 99.912% ( 4400, 6600 ] 4 0.088% 100.000% ** Level 5 read latency histogram (micros): Count: 37748 Average: 304.8461 StdDev: 495.16 Min: 73 Median: 158.0887 Max: 6544 Percentiles: P50: 158.09 P75: 243.95 P99: 2896.22 P99.9: 4387.88 P99.99: 6308.21 ------------------------------------------------------ ( 51, 76 ] 3 0.008% 0.008% ( 76, 110 ] 5096 13.500% 13.508% ### ( 110, 170 ] 17187 45.531% 59.039% ######### ( 170, 250 ] 6518 17.267% 76.306% ### ( 250, 380 ] 3167 8.390% 84.696% ## ( 380, 580 ] 2426 6.427% 91.123% # ( 580, 870 ] 1186 3.142% 94.265% # ( 870, 1300 ] 576 1.526% 95.791% ( 1300, 1900 ] 558 1.478% 97.269% ( 1900, 2900 ] 656 1.738% 99.007% ( 2900, 4400 ] 340 0.901% 99.907% ( 4400, 6600 ] 36 0.095% 100.003% ** DB Stats ** Uptime(secs): 4.5 total, 4.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 216.87 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.775 H:M:S, 17.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 224.72 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.775 H:M:S, 17.7 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/11/27-10:35:51 ... thread 0: (1000000,2000000) ops and (124797.2,161419.0) ops/second in (8.013001,12.390112) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/12 5.33 GB 194.7 2.8 0.0 2.8 4.7 1.9 0.0 2.4 169.1 285.2 16.76 15.68 39 0.430 2877K 6285 0.0 0.0 L1 2/2 228.34 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 18/0 978.03 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 156/0 9.63 GB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L5 1112/0 91.54 GB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1306/14 107.68 GB 0.0 2.8 0.0 2.8 4.7 1.9 0.0 2.4 169.1 285.2 16.76 15.68 39 0.430 2877K 6285 0.0 0.0 Int 0/0 0.00 KB 0.0 2.8 0.0 2.8 4.7 1.9 0.0 2.4 169.1 285.2 16.76 15.68 39 0.430 2877K 6285 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 2.8 0.0 2.8 2.8 -0.0 0.0 0.0 266.8 266.1 10.62 10.44 8 1.328 2877K 6285 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.9 1.9 0.0 0.0 0.0 318.1 6.14 5.24 31 0.198 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.6 total, 12.4 interval Flush(GB): cumulative 1.907, interval 1.907 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 4.67 GB write, 380.90 MB/s write, 2.77 GB read, 225.80 MB/s read, 16.8 seconds Interval compaction: 4.67 GB write, 385.79 MB/s write, 2.77 GB read, 228.70 MB/s read, 16.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 13, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 13, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 13, total-stops: 0, interval: 13 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 931265 Average: 17.1364 StdDev: 71.41 Min: 0 Median: 1.4317 Max: 9488 Percentiles: P50: 1.43 P75: 1.90 P99: 246.34 P99.9: 563.37 P99.99: 746.58 ------------------------------------------------------ [ 0, 1 ] 250676 26.918% 26.918% ##### ( 1, 2 ] 497924 53.467% 80.385% ########### ( 2, 3 ] 64145 6.888% 87.273% # ( 3, 4 ] 21113 2.267% 89.540% ( 4, 6 ] 13074 1.404% 90.944% ( 6, 10 ] 4790 0.514% 91.459% ( 10, 15 ] 944 0.101% 91.560% ( 15, 22 ] 153 0.016% 91.576% ( 22, 34 ] 50 0.005% 91.582% ( 34, 51 ] 22 0.002% 91.584% ( 51, 76 ] 24 0.003% 91.587% ( 76, 110 ] 12303 1.321% 92.908% ( 110, 170 ] 43163 4.635% 97.543% # ( 170, 250 ] 14222 1.527% 99.070% ( 250, 380 ] 4564 0.490% 99.560% ( 380, 580 ] 3454 0.371% 99.931% ( 580, 870 ] 959 0.103% 100.034% ( 870, 1300 ] 179 0.019% 100.053% ( 1300, 1900 ] 68 0.007% 100.060% ( 1900, 2900 ] 52 0.006% 100.066% ( 2900, 4400 ] 34 0.004% 100.070% ( 4400, 6600 ] 12 0.001% 100.071% ( 6600, 9900 ] 5 0.001% 100.071% ** Level 1 read latency histogram (micros): Count: 473 Average: 250.4651 StdDev: 644.33 Min: 80 Median: 147.9121 Max: 8165 Percentiles: P50: 147.91 P75: 189.19 P99: 3305.00 P99.9: 8165.00 P99.99: 8165.00 ------------------------------------------------------ ( 76, 110 ] 64 13.531% 13.531% ### ( 110, 170 ] 273 57.717% 71.247% ############ ( 170, 250 ] 74 15.645% 86.892% ### ( 250, 380 ] 20 4.228% 91.121% # ( 380, 580 ] 27 5.708% 96.829% # ( 580, 870 ] 5 1.057% 97.886% ( 870, 1300 ] 4 0.846% 98.732% ( 1900, 2900 ] 1 0.211% 98.943% ( 2900, 4400 ] 1 0.211% 99.154% ( 4400, 6600 ] 2 0.423% 99.577% ( 6600, 9900 ] 2 0.423% 100.000% ** Level 3 read latency histogram (micros): Count: 2204 Average: 206.5181 StdDev: 348.21 Min: 80 Median: 148.8907 Max: 8383 Percentiles: P50: 148.89 P75: 196.46 P99: 1083.57 P99.9: 6375.60 P99.99: 8383.00 ------------------------------------------------------ ( 76, 110 ] 319 14.474% 14.474% ### ( 110, 170 ] 1208 54.809% 69.283% ########### ( 170, 250 ] 381 17.287% 86.570% ### ( 250, 380 ] 135 6.125% 92.695% # ( 380, 580 ] 93 4.220% 96.915% # ( 580, 870 ] 40 1.815% 98.730% ( 870, 1300 ] 12 0.544% 99.274% ( 1300, 1900 ] 7 0.318% 99.592% ( 1900, 2900 ] 2 0.091% 99.682% ( 2900, 4400 ] 3 0.136% 99.819% ( 4400, 6600 ] 2 0.091% 99.909% ( 6600, 9900 ] 2 0.091% 100.000% ** Level 4 read latency histogram (micros): Count: 18105 Average: 198.8187 StdDev: 233.45 Min: 78 Median: 148.3002 Max: 5189 Percentiles: P50: 148.30 P75: 197.18 P99: 1157.73 P99.9: 3616.39 P99.99: 5189.00 ------------------------------------------------------ ( 76, 110 ] 2920 16.128% 16.128% ### ( 110, 170 ] 9607 53.063% 69.191% ########### ( 170, 250 ] 3096 17.100% 86.291% ### ( 250, 380 ] 1106 6.109% 92.400% # ( 380, 580 ] 842 4.651% 97.051% # ( 580, 870 ] 276 1.524% 98.575% ( 870, 1300 ] 115 0.635% 99.210% ( 1300, 1900 ] 74 0.409% 99.619% ( 1900, 2900 ] 38 0.210% 99.829% ( 2900, 4400 ] 27 0.149% 99.978% ( 4400, 6600 ] 4 0.022% 100.000% ** Level 5 read latency histogram (micros): Count: 152385 Average: 204.7722 StdDev: 271.24 Min: 73 Median: 148.1774 Max: 6544 Percentiles: P50: 148.18 P75: 196.32 P99: 1428.07 P99.9: 3875.49 P99.99: 5302.09 ------------------------------------------------------ ( 51, 76 ] 9 0.006% 0.006% ( 76, 110 ] 24394 16.008% 16.014% ### ( 110, 170 ] 81393 53.413% 69.427% ########### ( 170, 250 ] 25812 16.939% 86.365% ### ( 250, 380 ] 9052 5.940% 92.306% # ( 380, 580 ] 6925 4.544% 96.850% # ( 580, 870 ] 2416 1.585% 98.436% ( 870, 1300 ] 734 0.482% 98.917% ( 1300, 1900 ] 591 0.388% 99.305% ( 1900, 2900 ] 679 0.446% 99.751% ( 2900, 4400 ] 350 0.230% 99.980% ( 4400, 6600 ] 36 0.024% 100.004% ** DB Stats ** Uptime(secs): 12.6 total, 8.0 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 156.83 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:5.172 H:M:S, 41.2 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 122.83 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:4.397 H:M:S, 54.9 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 0 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 199.557 micros/op 20029 ops/sec 60.162 seconds 1204996 operations; 19.4 MB/s (313999 of 313999 found) Microseconds per read: Count: 1204996 Average: 199.5587 StdDev: 269.16 Min: 2 Median: 167.2085 Max: 35776 Percentiles: P50: 167.21 P75: 226.04 P99: 607.03 P99.9: 4003.98 P99.99: 9798.06 ------------------------------------------------------ ( 1, 2 ] 14 0.001% 0.001% ( 2, 3 ] 60 0.005% 0.006% ( 3, 4 ] 65 0.005% 0.012% ( 4, 6 ] 86 0.007% 0.019% ( 6, 10 ] 575 0.048% 0.066% ( 10, 15 ] 6669 0.553% 0.620% ( 15, 22 ] 2735 0.227% 0.847% ( 22, 34 ] 11922 0.989% 1.836% ( 34, 51 ] 1005 0.083% 1.920% ( 51, 76 ] 70 0.006% 1.925% ( 76, 110 ] 29385 2.439% 4.364% ( 110, 170 ] 576745 47.863% 52.227% ########## ( 170, 250 ] 391731 32.509% 84.736% ####### ( 250, 380 ] 133131 11.048% 95.784% ## ( 380, 580 ] 37902 3.145% 98.929% # ( 580, 870 ] 9129 0.758% 99.687% ( 870, 1300 ] 1694 0.141% 99.828% ( 1300, 1900 ] 315 0.026% 99.854% ( 1900, 2900 ] 335 0.028% 99.881% ( 2900, 4400 ] 303 0.025% 99.907% ( 4400, 6600 ] 581 0.048% 99.955% ( 6600, 9900 ] 437 0.036% 99.991% ( 9900, 14000 ] 64 0.005% 99.996% ( 14000, 22000 ] 21 0.002% 99.998% ( 22000, 33000 ] 21 0.002% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 21052717 rocksdb.bloom.filter.full.positive COUNT : 1409598 rocksdb.bloom.filter.full.true.positive COUNT : 1204554 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 442 rocksdb.memtable.miss COUNT : 1204554 rocksdb.l0.hit COUNT : 66498 rocksdb.l1.hit COUNT : 2657 rocksdb.l2andup.hit COUNT : 1135399 rocksdb.compaction.key.drop.new COUNT : 36710 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 2371429 rocksdb.number.keys.read COUNT : 1204996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 2447314728 rocksdb.bytes.read COUNT : 1204996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1348 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 50543886 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 6441176155 rocksdb.compact.write.bytes COUNT : 6413036659 rocksdb.flush.write.bytes COUNT : 2444303623 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 6 rocksdb.number.superversion_acquires COUNT : 351 rocksdb.number.superversion_releases COUNT : 328 rocksdb.number.superversion_cleanups COUNT : 85 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 2424404992 rocksdb.memtable.garbage.bytes.at.flush COUNT : 776192 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 12965902747 rocksdb.non.last.level.read.count COUNT : 2987287 rocksdb.block.checksum.compute.count COUNT : 2984597 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 166.364219 P95 : 370.261255 P99 : 605.232555 P100 : 35773.000000 COUNT : 1204996 SUM : 239432445 rocksdb.db.write.micros P50 : 1.561706 P95 : 5.671586 P99 : 1088.684291 P100 : 2717.000000 COUNT : 2371429 SUM : 56835122 rocksdb.compaction.times.micros P50 : 1067777.777778 P95 : 3915000.000000 P99 : 3972178.000000 P100 : 3972178.000000 COUNT : 11 SUM : 16130314 rocksdb.compaction.times.cpu_micros P50 : 1067777.777778 P95 : 3912171.000000 P99 : 3912171.000000 P100 : 3912171.000000 COUNT : 11 SUM : 15888773 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 775.192308 P95 : 1645.000000 P99 : 2524.000000 P100 : 2524.000000 COUNT : 37 SUM : 32260 rocksdb.compaction.outfile.sync.micros P50 : 948.181818 P95 : 2466.666667 P99 : 2517.000000 P100 : 2517.000000 COUNT : 26 SUM : 29624 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 155.000000 P95 : 365.375000 P99 : 439.000000 P100 : 439.000000 COUNT : 49 SUM : 8766 rocksdb.table.open.io.micros P50 : 4791.190865 P95 : 8798.571429 P99 : 13344.000000 P100 : 23945.000000 COUNT : 1348 SUM : 6610460 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.589153 P95 : 162.833801 P99 : 240.552516 P100 : 8402.000000 COUNT : 1562106 SUM : 53862840 rocksdb.read.block.get.micros P50 : 137.281394 P95 : 238.250099 P99 : 497.031628 P100 : 11559.000000 COUNT : 1421146 SUM : 205013499 rocksdb.write.raw.block.micros P50 : 0.555433 P95 : 1.674634 P99 : 3.561910 P100 : 12790.000000 COUNT : 2144889 SUM : 4514048 rocksdb.numfiles.in.singlecompaction P50 : 4.000000 P95 : 12.000000 P99 : 14.000000 P100 : 14.000000 COUNT : 12 SUM : 59 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.577289 P95 : 2117.077202 P99 : 2705.000000 P100 : 2705.000000 COUNT : 241563 SUM : 50565363 rocksdb.sst.read.micros P50 : 92.392397 P95 : 205.249607 P99 : 373.891178 P100 : 9488.000000 COUNT : 2987302 SUM : 249092823 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1204996 SUM : 1204996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 2371429 SUM : 2447314728 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 212285.714286 P95 : 259750.000000 P99 : 300822.000000 P100 : 300822.000000 COUNT : 37 SUM : 7454589 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 45 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0