[2024-12-03 15:19:45.801507] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 23.11.0 initialization... [2024-12-03 15:19:45.801775] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid1024883 ] Set seed to 1733235585794853 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-03 15:19:45.911624] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-03 15:19:45.947360] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-03 15:19:45.947406] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 1024883' to capture a snapshot of events at runtime. [2024-12-03 15:19:45.947419] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid1024883 for offline analysis/debug. [2024-12-03 15:19:45.947443] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Tue Dec 3 15:19:50 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/03-15:19:57 ... thread 0: (1000000,1000000) ops and (129523.7,129523.7) ops/second in (7.720594,7.720594) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/13 2.10 GB 50.4 0.3 0.0 0.3 1.2 0.9 0.0 1.4 37.4 142.1 8.42 3.92 12 0.702 319K 358 0.0 0.0 L1 8/8 969.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 15/1 1003.48 MB 0.9 0.8 0.6 0.1 0.8 0.6 0.0 1.2 113.8 113.0 6.85 3.66 2 3.427 392K 0 0.0 0.0 L4 173/4 9.64 GB 1.0 5.4 3.4 1.9 5.3 3.4 0.0 1.5 107.2 105.7 51.12 26.15 34 1.504 1098K 1204 0.0 0.0 L5 1120/0 92.28 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1333/26 105.94 GB 0.0 6.4 4.1 2.4 7.2 4.8 3.7 8.4 99.0 111.1 66.39 33.73 48 1.383 1810K 1562 0.0 0.0 Int 0/0 0.00 KB 0.0 6.4 4.1 2.4 7.2 4.8 3.7 8.4 99.0 111.1 66.39 33.73 48 1.383 1810K 1562 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 6.4 4.1 2.4 6.3 4.0 0.0 0.0 109.2 107.9 60.19 31.08 37 1.627 1810K 1562 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 142.2 6.20 2.65 11 0.564 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 7.9 total, 7.7 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 7.20 GB write, 930.61 MB/s write, 6.42 GB read, 829.84 MB/s read, 66.4 seconds Interval compaction: 7.20 GB write, 954.93 MB/s write, 6.42 GB read, 851.52 MB/s read, 66.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 0, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 0, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 14, total-stops: 11, interval: 25 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 307451 Average: 19.0088 StdDev: 219.41 Min: 1 Median: 1.8019 Max: 38293 Percentiles: P50: 1.80 P75: 2.85 P99: 448.06 P99.9: 2713.52 P99.99: 4343.21 ------------------------------------------------------ [ 0, 1 ] 23762 7.729% 7.729% ## ( 1, 2 ] 162064 52.712% 60.441% ########### ( 2, 3 ] 52799 17.173% 77.614% ### ( 3, 4 ] 26878 8.742% 86.356% ## ( 4, 6 ] 26946 8.764% 95.121% ## ( 6, 10 ] 7135 2.321% 97.441% ( 10, 15 ] 1349 0.439% 97.880% ( 15, 22 ] 166 0.054% 97.934% ( 22, 34 ] 20 0.007% 97.940% ( 34, 51 ] 12 0.004% 97.944% ( 51, 76 ] 11 0.004% 97.948% ( 76, 110 ] 151 0.049% 97.997% ( 110, 170 ] 918 0.299% 98.296% ( 170, 250 ] 940 0.306% 98.601% ( 250, 380 ] 895 0.291% 98.893% ( 380, 580 ] 977 0.318% 99.210% ( 580, 870 ] 880 0.286% 99.497% ( 870, 1300 ] 622 0.202% 99.699% ( 1300, 1900 ] 383 0.125% 99.823% ( 1900, 2900 ] 292 0.095% 99.918% ( 2900, 4400 ] 231 0.075% 99.993% ( 4400, 6600 ] 58 0.019% 100.012% ( 6600, 9900 ] 8 0.003% 100.015% ( 14000, 22000 ] 1 0.000% 100.015% ( 22000, 33000 ] 2 0.001% 100.016% ( 33000, 50000 ] 3 0.001% 100.017% ** Level 1 read latency histogram (micros): Count: 200 Average: 990.7100 StdDev: 1137.91 Min: 95 Median: 588.7879 Max: 6989 Percentiles: P50: 588.79 P75: 1300.00 P99: 5866.67 P99.9: 6989.00 P99.99: 6989.00 ------------------------------------------------------ ( 76, 110 ] 3 1.500% 1.500% ( 110, 170 ] 26 13.000% 14.500% ### ( 170, 250 ] 27 13.500% 28.000% ### ( 250, 380 ] 18 9.000% 37.000% ## ( 380, 580 ] 25 12.500% 49.500% ### ( 580, 870 ] 33 16.500% 66.000% ### ( 870, 1300 ] 18 9.000% 75.000% ## ( 1300, 1900 ] 22 11.000% 86.000% ## ( 1900, 2900 ] 12 6.000% 92.000% # ( 2900, 4400 ] 12 6.000% 98.000% # ( 4400, 6600 ] 3 1.500% 99.500% ( 6600, 9900 ] 1 0.500% 100.000% ** Level 2 read latency histogram (micros): Count: 130864 Average: 13.7291 StdDev: 157.62 Min: 1 Median: 1.9434 Max: 7369 Percentiles: P50: 1.94 P75: 3.08 P99: 14.05 P99.9: 2718.57 P99.99: 5635.21 ------------------------------------------------------ [ 0, 1 ] 4449 3.400% 3.400% # ( 1, 2 ] 64641 49.396% 52.795% ########## ( 2, 3 ] 27971 21.374% 74.169% #### ( 3, 4 ] 13518 10.330% 84.499% ## ( 4, 6 ] 14459 11.049% 95.548% ## ( 6, 10 ] 3790 2.896% 98.444% # ( 10, 15 ] 898 0.686% 99.130% ( 15, 22 ] 135 0.103% 99.234% ( 22, 34 ] 15 0.011% 99.245% ( 34, 51 ] 13 0.010% 99.255% ( 51, 76 ] 10 0.008% 99.263% ( 76, 110 ] 3 0.002% 99.265% ( 110, 170 ] 8 0.006% 99.271% ( 170, 250 ] 11 0.008% 99.279% ( 250, 380 ] 48 0.037% 99.316% ( 380, 580 ] 90 0.069% 99.385% ( 580, 870 ] 200 0.153% 99.538% ( 870, 1300 ] 221 0.169% 99.707% ( 1300, 1900 ] 159 0.122% 99.828% ( 1900, 2900 ] 115 0.088% 99.916% ( 2900, 4400 ] 84 0.064% 99.980% ( 4400, 6600 ] 23 0.018% 99.998% ( 6600, 9900 ] 3 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 777024 Average: 11.3534 StdDev: 145.50 Min: 1 Median: 1.8754 Max: 12213 Percentiles: P50: 1.88 P75: 2.94 P99: 9.97 P99.9: 805.23 P99.99: 1119.97 ------------------------------------------------------ [ 0, 1 ] 32714 4.210% 4.210% # ( 1, 2 ] 406438 52.307% 56.517% ########## ( 2, 3 ] 153268 19.725% 76.242% #### ( 3, 4 ] 77119 9.925% 86.167% ## ( 4, 6 ] 79100 10.180% 96.347% ## ( 6, 10 ] 20779 2.674% 99.021% # ( 10, 15 ] 4654 0.599% 99.620% ( 15, 22 ] 639 0.082% 99.702% ( 22, 34 ] 84 0.011% 99.713% ( 34, 51 ] 30 0.004% 99.717% ( 51, 76 ] 38 0.005% 99.722% ( 76, 110 ] 26 0.003% 99.725% ( 110, 170 ] 52 0.007% 99.732% ( 170, 250 ] 112 0.014% 99.746% ( 250, 380 ] 211 0.027% 99.773% ( 380, 580 ] 423 0.054% 99.828% ( 580, 870 ] 721 0.093% 99.921% ( 870, 1300 ] 926 0.119% 100.040% ( 1300, 1900 ] 709 0.091% 100.131% ( 1900, 2900 ] 519 0.067% 100.198% ( 2900, 4400 ] 471 0.061% 100.259% ( 4400, 6600 ] 132 0.017% 100.276% ( 6600, 9900 ] 13 0.002% 100.277% ( 9900, 14000 ] 1 0.000% 100.277% ** Level 4 read latency histogram (micros): Count: 469023 Average: 21.1374 StdDev: 220.56 Min: 1 Median: 1.8741 Max: 44855 Percentiles: P50: 1.87 P75: 2.97 P99: 456.73 P99.9: 2475.33 P99.99: 3393.11 ------------------------------------------------------ [ 0, 1 ] 24912 5.311% 5.311% # ( 1, 2 ] 239794 51.126% 56.438% ########## ( 2, 3 ] 89553 19.094% 75.531% #### ( 3, 4 ] 47173 10.058% 85.589% ## ( 4, 6 ] 45862 9.778% 95.367% ## ( 6, 10 ] 12092 2.578% 97.945% # ( 10, 15 ] 2569 0.548% 98.493% ( 15, 22 ] 384 0.082% 98.575% ( 22, 34 ] 53 0.011% 98.586% ( 34, 51 ] 45 0.010% 98.596% ( 51, 76 ] 38 0.008% 98.604% ( 76, 110 ] 82 0.017% 98.621% ( 110, 170 ] 371 0.079% 98.700% ( 170, 250 ] 514 0.110% 98.810% ( 250, 380 ] 562 0.120% 98.930% ( 380, 580 ] 857 0.183% 99.113% ( 580, 870 ] 1181 0.252% 99.364% ( 870, 1300 ] 1261 0.269% 99.633% ( 1300, 1900 ] 896 0.191% 99.824% ( 1900, 2900 ] 617 0.132% 99.956% ( 2900, 4400 ] 487 0.104% 100.060% ( 4400, 6600 ] 143 0.030% 100.090% ( 6600, 9900 ] 28 0.006% 100.096% ( 9900, 14000 ] 3 0.001% 100.097% ( 14000, 22000 ] 3 0.001% 100.097% ( 33000, 50000 ] 1 0.000% 100.098% ** Level 5 read latency histogram (micros): Count: 17928 Average: 1191.4864 StdDev: 1841.42 Min: 77 Median: 644.1312 Max: 88010 Percentiles: P50: 644.13 P75: 1444.58 P99: 6486.13 P99.9: 9851.06 P99.99: 48074.63 ------------------------------------------------------ ( 76, 110 ] 372 2.075% 2.075% ( 110, 170 ] 1860 10.375% 12.450% ## ( 170, 250 ] 1934 10.788% 23.237% ## ( 250, 380 ] 1935 10.793% 34.031% ## ( 380, 580 ] 2317 12.924% 46.954% ### ( 580, 870 ] 2469 13.772% 60.726% ### ( 870, 1300 ] 2192 12.227% 72.953% ## ( 1300, 1900 ] 1523 8.495% 81.448% ## ( 1900, 2900 ] 1229 6.855% 88.303% # ( 2900, 4400 ] 1308 7.296% 95.599% # ( 4400, 6600 ] 643 3.587% 99.186% # ( 6600, 9900 ] 130 0.725% 99.911% ( 9900, 14000 ] 5 0.028% 99.939% ( 14000, 22000 ] 2 0.011% 99.950% ( 22000, 33000 ] 1 0.006% 99.955% ( 33000, 50000 ] 7 0.039% 99.994% ( 50000, 75000 ] 1 0.006% 100.000% ( 75000, 110000 ] 2 0.011% 100.011% ** DB Stats ** Uptime(secs): 7.9 total, 7.7 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 124.20 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:3.733 H:M:S, 47.1 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 127.43 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:3.733 H:M:S, 48.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 3 num-running-flushes: 1 2024/12/03-15:20:02 ... thread 0: (1000000,2000000) ops and (232043.3,166249.2) ops/second in (4.309540,12.030134) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/9 3.08 GB 188.8 1.7 0.0 1.7 3.5 1.8 0.0 1.9 99.1 209.2 17.16 12.06 33 0.520 1727K 1910 0.0 0.0 L1 8/8 969.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 874.70 MB 0.9 0.8 0.6 0.1 0.8 0.6 0.0 1.2 113.8 113.0 6.85 3.66 2 3.427 392K 0 0.0 0.0 L4 173/0 9.76 GB 1.0 5.6 3.6 2.0 5.5 3.5 0.0 1.6 109.4 107.9 52.36 27.25 35 1.496 1179K 1204 0.0 0.0 L5 1120/0 92.28 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1333/17 106.92 GB 0.0 8.0 4.2 3.8 9.8 5.9 3.7 5.3 107.5 131.1 76.37 42.97 70 1.091 3299K 3114 0.0 0.0 Int 0/0 0.00 KB 0.0 8.0 4.2 3.8 9.8 5.9 3.7 5.3 107.5 131.1 76.37 42.97 70 1.091 3299K 3114 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 8.0 4.2 3.8 7.9 4.1 0.0 0.0 122.8 121.5 66.88 37.44 43 1.555 3299K 3114 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 1.8 1.8 0.0 0.0 0.0 199.2 9.49 5.52 27 0.351 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 12.2 total, 12.0 interval Flush(GB): cumulative 1.846, interval 1.846 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 9.78 GB write, 818.52 MB/s write, 8.02 GB read, 671.25 MB/s read, 76.4 seconds Interval compaction: 9.78 GB write, 832.24 MB/s write, 8.02 GB read, 682.51 MB/s read, 76.4 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 4, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 4, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 18, total-stops: 11, interval: 29 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 642327 Average: 18.6058 StdDev: 159.80 Min: 0 Median: 1.6101 Max: 38293 Percentiles: P50: 1.61 P75: 2.35 P99: 337.08 P99.9: 1386.94 P99.99: 2679.56 ------------------------------------------------------ [ 0, 1 ] 111211 17.314% 17.314% ### ( 1, 2 ] 344104 53.571% 70.885% ########### ( 2, 3 ] 75416 11.741% 82.626% ## ( 3, 4 ] 35097 5.464% 88.090% # ( 4, 6 ] 31726 4.939% 93.030% # ( 6, 10 ] 8290 1.291% 94.320% ( 10, 15 ] 1659 0.258% 94.578% ( 15, 22 ] 219 0.034% 94.613% ( 22, 34 ] 30 0.005% 94.617% ( 34, 51 ] 15 0.002% 94.620% ( 51, 76 ] 13 0.002% 94.622% ( 76, 110 ] 3737 0.582% 95.203% ( 110, 170 ] 15572 2.424% 97.628% ( 170, 250 ] 6769 1.054% 98.682% ( 250, 380 ] 3054 0.475% 99.157% ( 380, 580 ] 2602 0.405% 99.562% ( 580, 870 ] 1403 0.218% 99.780% ( 870, 1300 ] 710 0.111% 99.891% ( 1300, 1900 ] 398 0.062% 99.953% ( 1900, 2900 ] 305 0.047% 100.000% ( 2900, 4400 ] 240 0.037% 100.038% ( 4400, 6600 ] 59 0.009% 100.047% ( 6600, 9900 ] 8 0.001% 100.048% ( 14000, 22000 ] 1 0.000% 100.048% ( 22000, 33000 ] 2 0.000% 100.049% ( 33000, 50000 ] 3 0.000% 100.049% ** Level 1 read latency histogram (micros): Count: 875 Average: 370.0377 StdDev: 648.22 Min: 80 Median: 163.7467 Max: 6989 Percentiles: P50: 163.75 P75: 297.57 P99: 3806.25 P99.9: 6989.00 P99.99: 6989.00 ------------------------------------------------------ ( 76, 110 ] 98 11.200% 11.200% ## ( 110, 170 ] 379 43.314% 54.514% ######### ( 170, 250 ] 154 17.600% 72.114% #### ( 250, 380 ] 69 7.886% 80.000% ## ( 380, 580 ] 64 7.314% 87.314% # ( 580, 870 ] 40 4.571% 91.886% # ( 870, 1300 ] 21 2.400% 94.286% ( 1300, 1900 ] 22 2.514% 96.800% # ( 1900, 2900 ] 12 1.371% 98.171% ( 2900, 4400 ] 12 1.371% 99.543% ( 4400, 6600 ] 3 0.343% 99.886% ( 6600, 9900 ] 1 0.114% 100.000% ** Level 2 read latency histogram (micros): Count: 130864 Average: 13.7291 StdDev: 157.62 Min: 1 Median: 1.9434 Max: 7369 Percentiles: P50: 1.94 P75: 3.08 P99: 14.05 P99.9: 2718.57 P99.99: 5635.21 ------------------------------------------------------ [ 0, 1 ] 4449 3.400% 3.400% # ( 1, 2 ] 64641 49.396% 52.795% ########## ( 2, 3 ] 27971 21.374% 74.169% #### ( 3, 4 ] 13518 10.330% 84.499% ## ( 4, 6 ] 14459 11.049% 95.548% ## ( 6, 10 ] 3790 2.896% 98.444% # ( 10, 15 ] 898 0.686% 99.130% ( 15, 22 ] 135 0.103% 99.234% ( 22, 34 ] 15 0.011% 99.245% ( 34, 51 ] 13 0.010% 99.255% ( 51, 76 ] 10 0.008% 99.263% ( 76, 110 ] 3 0.002% 99.265% ( 110, 170 ] 8 0.006% 99.271% ( 170, 250 ] 11 0.008% 99.279% ( 250, 380 ] 48 0.037% 99.316% ( 380, 580 ] 90 0.069% 99.385% ( 580, 870 ] 200 0.153% 99.538% ( 870, 1300 ] 221 0.169% 99.707% ( 1300, 1900 ] 159 0.122% 99.828% ( 1900, 2900 ] 115 0.088% 99.916% ( 2900, 4400 ] 84 0.064% 99.980% ( 4400, 6600 ] 23 0.018% 99.998% ( 6600, 9900 ] 3 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 777757 Average: 11.5043 StdDev: 145.56 Min: 1 Median: 1.8763 Max: 12213 Percentiles: P50: 1.88 P75: 2.94 P99: 10.52 P99.9: 806.31 P99.99: 1120.13 ------------------------------------------------------ [ 0, 1 ] 32714 4.206% 4.206% # ( 1, 2 ] 406442 52.258% 56.464% ########## ( 2, 3 ] 153294 19.710% 76.174% #### ( 3, 4 ] 77132 9.917% 86.091% ## ( 4, 6 ] 79124 10.173% 96.265% ## ( 6, 10 ] 20785 2.672% 98.937% # ( 10, 15 ] 4655 0.599% 99.536% ( 15, 22 ] 639 0.082% 99.618% ( 22, 34 ] 84 0.011% 99.629% ( 34, 51 ] 30 0.004% 99.633% ( 51, 76 ] 38 0.005% 99.637% ( 76, 110 ] 106 0.014% 99.651% ( 110, 170 ] 399 0.051% 99.702% ( 170, 250 ] 234 0.030% 99.732% ( 250, 380 ] 271 0.035% 99.767% ( 380, 580 ] 461 0.059% 99.827% ( 580, 870 ] 732 0.094% 99.921% ( 870, 1300 ] 927 0.119% 100.040% ( 1300, 1900 ] 709 0.091% 100.131% ( 1900, 2900 ] 519 0.067% 100.198% ( 2900, 4400 ] 471 0.061% 100.258% ( 4400, 6600 ] 132 0.017% 100.275% ( 6600, 9900 ] 13 0.002% 100.277% ( 9900, 14000 ] 1 0.000% 100.277% ** Level 4 read latency histogram (micros): Count: 490248 Average: 23.3791 StdDev: 217.49 Min: 0 Median: 1.8839 Max: 44855 Percentiles: P50: 1.88 P75: 3.04 P99: 493.96 P99.9: 2442.41 P99.99: 3385.58 ------------------------------------------------------ [ 0, 1 ] 25916 5.286% 5.286% # ( 1, 2 ] 247991 50.585% 55.871% ########## ( 2, 3 ] 91712 18.707% 74.578% #### ( 3, 4 ] 48005 9.792% 84.370% ## ( 4, 6 ] 46658 9.517% 93.888% ## ( 6, 10 ] 12368 2.523% 96.410% # ( 10, 15 ] 2606 0.532% 96.942% ( 15, 22 ] 388 0.079% 97.021% ( 22, 34 ] 54 0.011% 97.032% ( 34, 51 ] 46 0.009% 97.041% ( 51, 76 ] 38 0.008% 97.049% ( 76, 110 ] 1145 0.234% 97.283% ( 110, 170 ] 4434 0.904% 98.187% ( 170, 250 ] 2085 0.425% 98.613% ( 250, 380 ] 1136 0.232% 98.844% ( 380, 580 ] 1340 0.273% 99.118% ( 580, 870 ] 1319 0.269% 99.387% ( 870, 1300 ] 1280 0.261% 99.648% ( 1300, 1900 ] 901 0.184% 99.832% ( 1900, 2900 ] 619 0.126% 99.958% ( 2900, 4400 ] 488 0.100% 100.057% ( 4400, 6600 ] 143 0.029% 100.086% ( 6600, 9900 ] 28 0.006% 100.092% ( 9900, 14000 ] 3 0.001% 100.093% ( 14000, 22000 ] 3 0.001% 100.093% ( 33000, 50000 ] 1 0.000% 100.094% ** Level 5 read latency histogram (micros): Count: 73473 Average: 434.1890 StdDev: 1013.07 Min: 75 Median: 166.2329 Max: 88010 Percentiles: P50: 166.23 P75: 331.27 P99: 4585.68 P99.9: 8441.07 P99.99: 34585.13 ------------------------------------------------------ ( 51, 76 ] 3 0.004% 0.004% ( 76, 110 ] 8245 11.222% 11.226% ## ( 110, 170 ] 30397 41.372% 52.598% ######## ( 170, 250 ] 12713 17.303% 69.901% ### ( 250, 380 ] 5993 8.157% 78.057% ## ( 380, 580 ] 5419 7.375% 85.433% # ( 580, 870 ] 3437 4.678% 90.111% # ( 870, 1300 ] 2361 3.213% 93.324% # ( 1300, 1900 ] 1558 2.121% 95.445% ( 1900, 2900 ] 1243 1.692% 97.136% ( 2900, 4400 ] 1315 1.790% 98.926% ( 4400, 6600 ] 643 0.875% 99.801% ( 6600, 9900 ] 130 0.177% 99.978% ( 9900, 14000 ] 5 0.007% 99.985% ( 14000, 22000 ] 2 0.003% 99.988% ( 22000, 33000 ] 1 0.001% 99.989% ( 33000, 50000 ] 7 0.010% 99.999% ( 50000, 75000 ] 1 0.001% 100.000% ( 75000, 110000 ] 2 0.003% 100.003% ** DB Stats ** Uptime(secs): 12.2 total, 4.3 interval Cumulative writes: 0 writes, 2000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 1.92 GB, 160.90 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:4.607 H:M:S, 37.7 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 228.38 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.875 H:M:S, 20.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 2024/12/03-15:20:12 ... thread 0: (1000000,3000000) ops and (96745.8,134129.1) ops/second in (10.336369,22.366503) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 18/13 3.99 GB 257.0 3.9 0.0 3.9 6.7 2.8 0.0 2.4 140.3 238.5 28.73 23.11 52 0.552 4092K 13K 0.0 0.0 L1 8/8 969.78 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 14/0 874.70 MB 0.9 0.8 0.6 0.1 0.8 0.6 0.0 1.2 113.8 113.0 6.85 3.66 2 3.427 392K 0 0.0 0.0 L4 173/0 9.76 GB 1.0 5.6 3.6 2.0 5.5 3.5 0.0 1.6 109.4 107.9 52.36 27.25 35 1.496 1179K 1204 0.0 0.0 L5 1120/0 92.28 GB 0.9 0.0 0.0 0.0 0.0 0.0 3.7 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1333/21 107.83 GB 0.0 10.3 4.2 6.1 13.0 6.9 3.7 4.7 119.9 151.0 87.94 54.02 89 0.988 5664K 14K 0.0 0.0 Int 0/0 0.00 KB 0.0 10.3 4.2 6.1 13.0 6.9 3.7 4.7 119.9 151.0 87.94 54.02 89 0.988 5664K 14K 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 10.3 4.2 6.1 10.2 4.1 0.0 0.0 139.6 138.3 75.51 45.94 47 1.607 5664K 14K 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 2.8 2.8 0.0 0.0 0.0 228.0 12.44 8.07 42 0.296 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 22.6 total, 22.4 interval Flush(GB): cumulative 2.769, interval 2.769 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 12.97 GB write, 588.23 MB/s write, 10.29 GB read, 467.05 MB/s read, 87.9 seconds Interval compaction: 12.97 GB write, 593.54 MB/s write, 10.29 GB read, 471.26 MB/s read, 87.9 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 14, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 14, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 11, pending-compaction-bytes-delays: 14, pending-compaction-bytes-stops: 0, total-delays: 28, total-stops: 11, interval: 39 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 1365506 Average: 19.1003 StdDev: 119.27 Min: 0 Median: 1.4909 Max: 38293 Percentiles: P50: 1.49 P75: 1.98 P99: 277.42 P99.9: 827.93 P99.99: 1496.30 ------------------------------------------------------ [ 0, 1 ] 339362 24.852% 24.852% ##### ( 1, 2 ] 699523 51.228% 76.081% ########## ( 2, 3 ] 111710 8.181% 84.261% ## ( 3, 4 ] 50645 3.709% 87.970% # ( 4, 6 ] 41228 3.019% 90.990% # ( 6, 10 ] 10068 0.737% 91.727% ( 10, 15 ] 2113 0.155% 91.882% ( 15, 22 ] 289 0.021% 91.903% ( 22, 34 ] 34 0.002% 91.905% ( 34, 51 ] 17 0.001% 91.907% ( 51, 76 ] 19 0.001% 91.908% ( 76, 110 ] 15815 1.158% 93.066% ( 110, 170 ] 59974 4.392% 97.458% # ( 170, 250 ] 19647 1.439% 98.897% ( 250, 380 ] 6675 0.489% 99.386% ( 380, 580 ] 5269 0.386% 99.772% ( 580, 870 ] 2051 0.150% 99.922% ( 870, 1300 ] 796 0.058% 99.980% ( 1300, 1900 ] 414 0.030% 100.010% ( 1900, 2900 ] 336 0.025% 100.035% ( 2900, 4400 ] 262 0.019% 100.054% ( 4400, 6600 ] 60 0.004% 100.059% ( 6600, 9900 ] 8 0.001% 100.059% ( 14000, 22000 ] 1 0.000% 100.059% ( 22000, 33000 ] 2 0.000% 100.059% ( 33000, 50000 ] 3 0.000% 100.060% ** Level 1 read latency histogram (micros): Count: 2542 Average: 235.8127 StdDev: 399.19 Min: 80 Median: 149.6875 Max: 6989 Percentiles: P50: 149.69 P75: 204.14 P99: 2115.00 P99.9: 5469.20 P99.99: 6989.00 ------------------------------------------------------ ( 76, 110 ] 382 15.028% 15.028% ### ( 110, 170 ] 1344 52.872% 67.899% ########### ( 170, 250 ] 423 16.640% 84.540% ### ( 250, 380 ] 144 5.665% 90.205% # ( 380, 580 ] 127 4.996% 95.201% # ( 580, 870 ] 50 1.967% 97.168% ( 870, 1300 ] 22 0.865% 98.033% ( 1300, 1900 ] 22 0.865% 98.899% ( 1900, 2900 ] 12 0.472% 99.371% ( 2900, 4400 ] 12 0.472% 99.843% ( 4400, 6600 ] 3 0.118% 99.961% ( 6600, 9900 ] 1 0.039% 100.000% ** Level 2 read latency histogram (micros): Count: 130864 Average: 13.7291 StdDev: 157.62 Min: 1 Median: 1.9434 Max: 7369 Percentiles: P50: 1.94 P75: 3.08 P99: 14.05 P99.9: 2718.57 P99.99: 5635.21 ------------------------------------------------------ [ 0, 1 ] 4449 3.400% 3.400% # ( 1, 2 ] 64641 49.396% 52.795% ########## ( 2, 3 ] 27971 21.374% 74.169% #### ( 3, 4 ] 13518 10.330% 84.499% ## ( 4, 6 ] 14459 11.049% 95.548% ## ( 6, 10 ] 3790 2.896% 98.444% # ( 10, 15 ] 898 0.686% 99.130% ( 15, 22 ] 135 0.103% 99.234% ( 22, 34 ] 15 0.011% 99.245% ( 34, 51 ] 13 0.010% 99.255% ( 51, 76 ] 10 0.008% 99.263% ( 76, 110 ] 3 0.002% 99.265% ( 110, 170 ] 8 0.006% 99.271% ( 170, 250 ] 11 0.008% 99.279% ( 250, 380 ] 48 0.037% 99.316% ( 380, 580 ] 90 0.069% 99.385% ( 580, 870 ] 200 0.153% 99.538% ( 870, 1300 ] 221 0.169% 99.707% ( 1300, 1900 ] 159 0.122% 99.828% ( 1900, 2900 ] 115 0.088% 99.916% ( 2900, 4400 ] 84 0.064% 99.980% ( 4400, 6600 ] 23 0.018% 99.998% ( 6600, 9900 ] 3 0.002% 100.000% ** Level 3 read latency histogram (micros): Count: 779759 Average: 11.8725 StdDev: 145.68 Min: 1 Median: 1.8787 Max: 12213 Percentiles: P50: 1.88 P75: 2.95 P99: 12.49 P99.9: 808.25 P99.99: 1120.69 ------------------------------------------------------ [ 0, 1 ] 32714 4.195% 4.195% # ( 1, 2 ] 406449 52.125% 56.320% ########## ( 2, 3 ] 153334 19.664% 75.985% #### ( 3, 4 ] 77164 9.896% 85.881% ## ( 4, 6 ] 79190 10.156% 96.036% ## ( 6, 10 ] 20794 2.667% 98.703% # ( 10, 15 ] 4656 0.597% 99.300% ( 15, 22 ] 639 0.082% 99.382% ( 22, 34 ] 84 0.011% 99.393% ( 34, 51 ] 30 0.004% 99.397% ( 51, 76 ] 38 0.005% 99.401% ( 76, 110 ] 431 0.055% 99.457% ( 110, 170 ] 1474 0.189% 99.646% ( 170, 250 ] 531 0.068% 99.714% ( 250, 380 ] 337 0.043% 99.757% ( 380, 580 ] 520 0.067% 99.824% ( 580, 870 ] 755 0.097% 99.921% ( 870, 1300 ] 928 0.119% 100.040% ( 1300, 1900 ] 709 0.091% 100.131% ( 1900, 2900 ] 519 0.067% 100.197% ( 2900, 4400 ] 472 0.061% 100.258% ( 4400, 6600 ] 132 0.017% 100.275% ( 6600, 9900 ] 13 0.002% 100.276% ( 9900, 14000 ] 1 0.000% 100.276% ** Level 4 read latency histogram (micros): Count: 515006 Average: 29.6893 StdDev: 215.46 Min: 0 Median: 1.9335 Max: 44855 Percentiles: P50: 1.93 P75: 3.41 P99: 524.30 P99.9: 2409.64 P99.99: 3385.20 ------------------------------------------------------ [ 0, 1 ] 25916 5.032% 5.032% # ( 1, 2 ] 248082 48.171% 53.203% ########## ( 2, 3 ] 92312 17.924% 71.127% #### ( 3, 4 ] 48419 9.402% 80.529% ## ( 4, 6 ] 47383 9.200% 89.729% ## ( 6, 10 ] 12453 2.418% 92.147% ( 10, 15 ] 2629 0.510% 92.658% ( 15, 22 ] 393 0.076% 92.734% ( 22, 34 ] 54 0.010% 92.745% ( 34, 51 ] 46 0.009% 92.754% ( 51, 76 ] 39 0.008% 92.761% ( 76, 110 ] 5003 0.971% 93.733% ( 110, 170 ] 17650 3.427% 97.160% # ( 170, 250 ] 5706 1.108% 98.268% ( 250, 380 ] 2206 0.428% 98.696% ( 380, 580 ] 2169 0.421% 99.117% ( 580, 870 ] 1501 0.291% 99.409% ( 870, 1300 ] 1308 0.254% 99.663% ( 1300, 1900 ] 905 0.176% 99.838% ( 1900, 2900 ] 622 0.121% 99.959% ( 2900, 4400 ] 490 0.095% 100.054% ( 4400, 6600 ] 144 0.028% 100.082% ( 6600, 9900 ] 28 0.005% 100.088% ( 9900, 14000 ] 3 0.001% 100.088% ( 14000, 22000 ] 3 0.001% 100.089% ( 33000, 50000 ] 1 0.000% 100.089% ** Level 5 read latency histogram (micros): Count: 222764 Average: 255.0869 StdDev: 601.77 Min: 75 Median: 149.8960 Max: 88010 Percentiles: P50: 149.90 P75: 206.33 P99: 2809.51 P99.9: 6327.09 P99.99: 9664.52 ------------------------------------------------------ ( 51, 76 ] 11 0.005% 0.005% ( 76, 110 ] 33998 15.262% 15.267% ### ( 110, 170 ] 116362 52.236% 67.502% ########## ( 170, 250 ] 36783 16.512% 84.014% ### ( 250, 380 ] 12740 5.719% 89.734% # ( 380, 580 ] 10735 4.819% 94.553% # ( 580, 870 ] 4658 2.091% 96.644% ( 870, 1300 ] 2501 1.123% 97.766% ( 1300, 1900 ] 1587 0.712% 98.479% ( 1900, 2900 ] 1278 0.574% 99.052% ( 2900, 4400 ] 1326 0.595% 99.648% ( 4400, 6600 ] 643 0.289% 99.936% ( 6600, 9900 ] 130 0.058% 99.995% ( 9900, 14000 ] 6 0.003% 99.997% ( 14000, 22000 ] 2 0.001% 99.998% ( 22000, 33000 ] 1 0.000% 99.999% ( 33000, 50000 ] 7 0.003% 100.002% ( 50000, 75000 ] 1 0.000% 100.002% ( 75000, 110000 ] 2 0.001% 100.003% ** DB Stats ** Uptime(secs): 22.6 total, 10.3 interval Cumulative writes: 0 writes, 3000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 2.88 GB, 130.82 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:11.233 H:M:S, 49.8 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 95.22 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:6.626 H:M:S, 64.1 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 2 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 224.079 micros/op 17831 ops/sec 60.174 seconds 1072996 operations; 17.3 MB/s (263999 of 263999 found) Microseconds per read: Count: 1072996 Average: 224.0807 StdDev: 719.88 Min: 2 Median: 169.2619 Max: 208970 Percentiles: P50: 169.26 P75: 229.23 P99: 819.37 P99.9: 9950.91 P99.99: 27152.24 ------------------------------------------------------ ( 1, 2 ] 4 0.000% 0.000% ( 2, 3 ] 27 0.003% 0.003% ( 3, 4 ] 55 0.005% 0.008% ( 4, 6 ] 89 0.008% 0.016% ( 6, 10 ] 387 0.036% 0.052% ( 10, 15 ] 4251 0.396% 0.449% ( 15, 22 ] 4739 0.442% 0.890% ( 22, 34 ] 11324 1.055% 1.946% ( 34, 51 ] 2745 0.256% 2.201% ( 51, 76 ] 364 0.034% 2.235% ( 76, 110 ] 25560 2.382% 4.617% ( 110, 170 ] 493018 45.948% 50.565% ######### ( 170, 250 ] 354138 33.005% 83.570% ####### ( 250, 380 ] 109686 10.222% 93.792% ## ( 380, 580 ] 44217 4.121% 97.913% # ( 580, 870 ] 14129 1.317% 99.230% ( 870, 1300 ] 3976 0.371% 99.600% ( 1300, 1900 ] 1231 0.115% 99.715% ( 1900, 2900 ] 892 0.083% 99.798% ( 2900, 4400 ] 648 0.060% 99.859% ( 4400, 6600 ] 246 0.023% 99.882% ( 6600, 9900 ] 192 0.018% 99.900% ( 9900, 14000 ] 403 0.038% 99.937% ( 14000, 22000 ] 504 0.047% 99.984% ( 22000, 33000 ] 136 0.013% 99.997% ( 33000, 50000 ] 22 0.002% 99.999% ( 50000, 75000 ] 9 0.001% 100.000% ( 110000, 170000 ] 1 0.000% 100.000% ( 170000, 250000 ] 3 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 14917135 rocksdb.bloom.filter.full.positive COUNT : 1217869 rocksdb.bloom.filter.full.true.positive COUNT : 1072556 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 440 rocksdb.memtable.miss COUNT : 1072556 rocksdb.l0.hit COUNT : 43216 rocksdb.l1.hit COUNT : 15112 rocksdb.l2andup.hit COUNT : 1014228 rocksdb.compaction.key.drop.new COUNT : 148172 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 3834438 rocksdb.number.keys.read COUNT : 1072996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 3957140016 rocksdb.bytes.read COUNT : 1072996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1536 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 44960027 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 16919342983 rocksdb.compact.write.bytes COUNT : 16762687893 rocksdb.flush.write.bytes COUNT : 3897525591 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 38 rocksdb.number.superversion_acquires COUNT : 724 rocksdb.number.superversion_releases COUNT : 693 rocksdb.number.superversion_cleanups COUNT : 181 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 203 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 3865944064 rocksdb.memtable.garbage.bytes.at.flush COUNT : 1366016 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 21418147545 rocksdb.non.last.level.read.count COUNT : 5031549 rocksdb.block.checksum.compute.count COUNT : 5325060 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 168.266977 P95 : 436.940479 P99 : 818.532897 P100 : 208968.000000 COUNT : 1072996 SUM : 239420774 rocksdb.db.write.micros P50 : 1.530831 P95 : 5.192951 P99 : 15.278277 P100 : 720951.000000 COUNT : 3834438 SUM : 54964316 rocksdb.compaction.times.micros P50 : 1426470.588235 P95 : 5290000.000000 P99 : 40520000.000000 P100 : 43103358.000000 COUNT : 53 SUM : 127280305 rocksdb.compaction.times.cpu_micros P50 : 951071.428571 P95 : 4685000.000000 P99 : 16783090.000000 P100 : 16783090.000000 COUNT : 53 SUM : 71301149 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 729.142857 P95 : 82000.000000 P99 : 296196.000000 P100 : 296196.000000 COUNT : 56 SUM : 623550 rocksdb.compaction.outfile.sync.micros P50 : 20000.000000 P95 : 228545.454545 P99 : 360825.000000 P100 : 556005.000000 COUNT : 159 SUM : 9630499 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 161.951220 P95 : 3875.000000 P99 : 5633.000000 P100 : 5633.000000 COUNT : 123 SUM : 111816 rocksdb.table.open.io.micros P50 : 9123.956044 P95 : 20221.428571 P99 : 28402.000000 P100 : 188802.000000 COUNT : 1536 SUM : 14970642 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.809787 P95 : 108.588721 P99 : 223.898007 P100 : 38299.000000 COUNT : 4096623 SUM : 75315356 rocksdb.read.block.get.micros P50 : 140.672725 P95 : 317.529108 P99 : 738.208591 P100 : 88155.000000 COUNT : 1226996 SUM : 203036413 rocksdb.write.raw.block.micros P50 : 0.539932 P95 : 1.459517 P99 : 3.630853 P100 : 78766.000000 COUNT : 5003965 SUM : 13248791 rocksdb.numfiles.in.singlecompaction P50 : 3.300000 P95 : 5.883333 P99 : 25.000000 P100 : 25.000000 COUNT : 67 SUM : 274 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.570539 P95 : 1236.257126 P99 : 2635.439484 P100 : 720760.000000 COUNT : 251909 SUM : 44985351 rocksdb.sst.read.micros P50 : 2.020969 P95 : 183.525084 P99 : 456.465573 P100 : 88010.000000 COUNT : 5031763 SUM : 261506569 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1072996 SUM : 1072996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 3834438 SUM : 3957140016 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 219777.777778 P95 : 744000.000000 P99 : 1006216.000000 P100 : 1006216.000000 COUNT : 56 SUM : 15276648 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 205 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0