[2024-12-09 01:54:39.346340] Starting SPDK v24.01.1-pre git sha1 c13c99a5e / DPDK 22.11.4 initialization... [2024-12-09 01:54:39.346603] [ DPDK EAL parameters: rocksdb --no-shconf -c 0x1 --huge-unlink --no-telemetry --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6 --base-virtaddr=0x200000000000 --match-allocations --file-prefix=spdk_pid231310 ] Set seed to 1733705679339682 because --seed was 0 EAL: No free 2048 kB hugepages reported on node 1 [2024-12-09 01:54:39.444366] app.c: 798:spdk_app_start: *NOTICE*: Total cores available: 1 [2024-12-09 01:54:39.480717] app.c: 488:app_setup_trace: *NOTICE*: Tracepoint Group Mask 0x80 specified. [2024-12-09 01:54:39.480761] app.c: 489:app_setup_trace: *NOTICE*: Use 'spdk_trace -s rocksdb -p 231310' to capture a snapshot of events at runtime. [2024-12-09 01:54:39.480773] app.c: 494:app_setup_trace: *NOTICE*: Or copy /dev/shm/rocksdb_trace.pid231310 for offline analysis/debug. [2024-12-09 01:54:39.480799] reactor.c: 937:reactor_run: *NOTICE*: Reactor started on core 0 RocksDB: version 8.1.1 Date: Mon Dec 9 01:54:43 2024 CPU: 96 * Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz CPUCache: 36608 KB 2024/12/09-01:54:48 ... thread 0: (1000000,1000000) ops and (224967.1,224967.1) ops/second in (4.445094,4.445094) seconds ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ L0 17/12 3.66 GB 88.1 0.7 0.0 0.7 1.6 0.9 0.0 1.9 129.0 279.3 5.86 5.12 17 0.345 767K 728 0.0 0.0 L1 10/10 1.23 GB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L2 2/0 86.95 MB 0.9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L3 24/0 970.67 MB 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 L4 162/0 9.61 GB 1.0 0.9 0.4 0.5 0.9 0.4 0.0 2.2 148.7 148.1 5.92 4.27 5 1.184 0 0 0.0 0.0 L5 1116/0 91.78 GB 0.9 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 Sum 1331/22 107.31 GB 0.0 1.6 0.4 1.2 2.5 1.2 1.0 2.9 138.9 213.4 11.78 9.40 22 0.536 767K 728 0.0 0.0 Int 0/0 0.00 KB 0.0 1.6 0.4 1.2 2.5 1.2 1.0 2.9 138.9 213.4 11.78 9.40 22 0.536 767K 728 0.0 0.0 ** Compaction Stats [default] ** Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Low 0/0 0.00 KB 0.0 1.6 0.4 1.2 1.6 0.4 0.0 0.0 186.5 185.9 8.78 7.06 8 1.097 767K 728 0.0 0.0 High 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.9 0.9 0.0 0.0 0.0 293.5 3.01 2.33 14 0.215 0 0 0.0 0.0 Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 Uptime(secs): 4.6 total, 4.4 interval Flush(GB): cumulative 0.861, interval 0.861 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 2.46 GB write, 541.16 MB/s write, 1.60 GB read, 352.33 MB/s read, 11.8 seconds Interval compaction: 2.46 GB write, 565.40 MB/s write, 1.60 GB read, 368.12 MB/s read, 11.8 seconds Write Stall (count): cf-l0-file-count-limit-delays-with-ongoing-compaction: 3, cf-l0-file-count-limit-stops-with-ongoing-compaction: 0, l0-file-count-limit-delays: 3, l0-file-count-limit-stops: 0, memtable-limit-delays: 0, memtable-limit-stops: 0, pending-compaction-bytes-delays: 0, pending-compaction-bytes-stops: 0, total-delays: 3, total-stops: 0, interval: 3 total count ** File Read Latency Histogram By Level [default] ** ** Level 0 read latency histogram (micros): Count: 338802 Average: 11.2228 StdDev: 85.84 Min: 1 Median: 1.5039 Max: 9697 Percentiles: P50: 1.50 P75: 1.94 P99: 214.69 P99.9: 785.65 P99.99: 1249.29 ------------------------------------------------------ [ 0, 1 ] 70689 20.864% 20.864% #### ( 1, 2 ] 195893 57.819% 78.684% ############ ( 2, 3 ] 35716 10.542% 89.226% ## ( 3, 4 ] 11887 3.509% 92.734% # ( 4, 6 ] 8386 2.475% 95.209% ( 6, 10 ] 3049 0.900% 96.109% ( 10, 15 ] 665 0.196% 96.306% ( 15, 22 ] 111 0.033% 96.338% ( 22, 34 ] 46 0.014% 96.352% ( 34, 51 ] 12 0.004% 96.355% ( 51, 76 ] 7 0.002% 96.357% ( 76, 110 ] 1576 0.465% 96.823% ( 110, 170 ] 6093 1.798% 98.621% ( 170, 250 ] 2300 0.679% 99.300% ( 250, 380 ] 914 0.270% 99.570% ( 380, 580 ] 794 0.234% 99.804% ( 580, 870 ] 460 0.136% 99.940% ( 870, 1300 ] 194 0.057% 99.997% ( 1300, 1900 ] 100 0.030% 100.027% ( 1900, 2900 ] 49 0.014% 100.041% ( 2900, 4400 ] 18 0.005% 100.046% ( 4400, 6600 ] 12 0.004% 100.050% ( 6600, 9900 ] 9 0.003% 100.053% ** Level 1 read latency histogram (micros): Count: 438 Average: 392.4680 StdDev: 792.77 Min: 81 Median: 156.8293 Max: 6521 Percentiles: P50: 156.83 P75: 239.73 P99: 4672.80 P99.9: 6407.28 P99.99: 6521.00 ------------------------------------------------------ ( 76, 110 ] 59 13.470% 13.470% ### ( 110, 170 ] 205 46.804% 60.274% ######### ( 170, 250 ] 74 16.895% 77.169% ### ( 250, 380 ] 35 7.991% 85.160% ## ( 380, 580 ] 19 4.338% 89.498% # ( 580, 870 ] 8 1.826% 91.324% ( 870, 1300 ] 8 1.826% 93.151% ( 1300, 1900 ] 7 1.598% 94.749% ( 1900, 2900 ] 12 2.740% 97.489% # ( 2900, 4400 ] 6 1.370% 98.858% ( 4400, 6600 ] 5 1.142% 100.000% ** Level 2 read latency histogram (micros): Count: 50 Average: 313.6400 StdDev: 391.25 Min: 83 Median: 167.0000 Max: 2120 Percentiles: P50: 167.00 P75: 289.00 P99: 2120.00 P99.9: 2120.00 P99.99: 2120.00 ------------------------------------------------------ ( 76, 110 ] 6 12.000% 12.000% ## ( 110, 170 ] 20 40.000% 52.000% ######## ( 170, 250 ] 10 20.000% 72.000% #### ( 250, 380 ] 5 10.000% 82.000% ## ( 380, 580 ] 2 4.000% 86.000% # ( 580, 870 ] 4 8.000% 94.000% ## ( 870, 1300 ] 1 2.000% 96.000% ( 1300, 1900 ] 1 2.000% 98.000% ( 1900, 2900 ] 1 2.000% 100.000% ** Level 3 read latency histogram (micros): Count: 561 Average: 315.1676 StdDev: 371.04 Min: 81 Median: 167.1250 Max: 2754 Percentiles: P50: 167.12 P75: 336.88 P99: 2276.67 P99.9: 2754.00 P99.99: 2754.00 ------------------------------------------------------ ( 76, 110 ] 52 9.269% 9.269% ## ( 110, 170 ] 240 42.781% 52.050% ######### ( 170, 250 ] 94 16.756% 68.806% ### ( 250, 380 ] 52 9.269% 78.075% ## ( 380, 580 ] 53 9.447% 87.522% ## ( 580, 870 ] 30 5.348% 92.870% # ( 870, 1300 ] 21 3.743% 96.613% # ( 1300, 1900 ] 10 1.783% 98.396% ( 1900, 2900 ] 9 1.604% 100.000% ** Level 4 read latency histogram (micros): Count: 67733 Average: 29.6449 StdDev: 180.56 Min: 1 Median: 1.8142 Max: 5748 Percentiles: P50: 1.81 P75: 2.89 P99: 677.86 P99.9: 2236.40 P99.99: 2991.32 ------------------------------------------------------ [ 0, 1 ] 5941 8.771% 8.771% ## ( 1, 2 ] 34296 50.634% 59.405% ########## ( 2, 3 ] 11861 17.511% 76.917% #### ( 3, 4 ] 4923 7.268% 84.185% # ( 4, 6 ] 4425 6.533% 90.718% # ( 6, 10 ] 1631 2.408% 93.126% ( 10, 15 ] 333 0.492% 93.618% ( 15, 22 ] 68 0.100% 93.718% ( 22, 34 ] 20 0.030% 93.748% ( 34, 51 ] 6 0.009% 93.756% ( 51, 76 ] 3 0.004% 93.761% ( 76, 110 ] 410 0.605% 94.366% ( 110, 170 ] 1508 2.226% 96.593% ( 170, 250 ] 615 0.908% 97.500% ( 250, 380 ] 423 0.625% 98.125% ( 380, 580 ] 483 0.713% 98.838% ( 580, 870 ] 325 0.480% 99.318% ( 870, 1300 ] 232 0.343% 99.660% ( 1300, 1900 ] 133 0.196% 99.857% ( 1900, 2900 ] 87 0.128% 99.985% ( 2900, 4400 ] 53 0.078% 100.063% ( 4400, 6600 ] 6 0.009% 100.072% ** Level 5 read latency histogram (micros): Count: 30747 Average: 386.5638 StdDev: 657.15 Min: 75 Median: 163.9832 Max: 7478 Percentiles: P50: 163.98 P75: 312.89 P99: 3841.40 P99.9: 6023.10 P99.99: 7108.91 ------------------------------------------------------ ( 51, 76 ] 1 0.003% 0.003% ( 76, 110 ] 3794 12.339% 12.343% ## ( 110, 170 ] 12869 41.854% 54.197% ######## ( 170, 250 ] 5107 16.610% 70.807% ### ( 250, 380 ] 2665 8.668% 79.474% ## ( 380, 580 ] 2317 7.536% 87.010% ## ( 580, 870 ] 1245 4.049% 91.059% # ( 870, 1300 ] 782 2.543% 93.603% # ( 1300, 1900 ] 612 1.990% 95.593% ( 1900, 2900 ] 708 2.303% 97.896% ( 2900, 4400 ] 541 1.760% 99.655% ( 4400, 6600 ] 102 0.332% 99.987% ( 6600, 9900 ] 6 0.020% 100.007% ** DB Stats ** Uptime(secs): 4.6 total, 4.4 interval Cumulative writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.96 GB, 211.84 MB/s Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Cumulative stall: 00:00:0.770 H:M:S, 16.6 percent Interval writes: 0 writes, 1000K keys, 0 commit groups, 0.0 writes per commit group, ingest: 984.19 MB, 221.28 MB/s Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Interval stall: 00:00:0.770 H:M:S, 17.3 percent Write Stall (count): write-buffer-manager-limit-stops: 0, num-running-compactions: 1 num-running-flushes: 1 using bdev Nvme0n1 Initializing RocksDB Options from the specified file Initializing RocksDB Options from command-line flags Integrated BlobDB: blob cache disabled Keys: 16 bytes each (+ 0 bytes user-defined timestamp) Values: 1000 bytes each (1000 bytes after compression) Entries: 100000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 96893.3 MB (estimated) FileSize: 96893.3 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Compression sampling rate: 0 Memtablerep: SkipListFactory Perf Level: 1 ------------------------------------------------ DB path: [/mnt/rocksdb] readwhilewriting : 193.194 micros/op 20688 ops/sec 60.131 seconds 1243996 operations; 20.0 MB/s (302999 of 302999 found) Microseconds per read: Count: 1243996 Average: 193.1956 StdDev: 318.25 Min: 2 Median: 162.6270 Max: 51523 Percentiles: P50: 162.63 P75: 219.57 P99: 552.85 P99.9: 4071.46 P99.99: 14743.89 ------------------------------------------------------ ( 1, 2 ] 15 0.001% 0.001% ( 2, 3 ] 48 0.004% 0.005% ( 3, 4 ] 61 0.005% 0.010% ( 4, 6 ] 69 0.006% 0.016% ( 6, 10 ] 830 0.067% 0.082% ( 10, 15 ] 5824 0.468% 0.550% ( 15, 22 ] 3131 0.252% 0.802% ( 22, 34 ] 7155 0.575% 1.377% ( 34, 51 ] 2621 0.211% 1.588% ( 51, 76 ] 118 0.009% 1.597% ( 76, 110 ] 38148 3.067% 4.664% # ( 110, 170 ] 642991 51.688% 56.352% ########## ( 170, 250 ] 374370 30.094% 86.446% ###### ( 250, 380 ] 130122 10.460% 96.906% ## ( 380, 580 ] 30145 2.423% 99.329% ( 580, 870 ] 5328 0.428% 99.757% ( 870, 1300 ] 1016 0.082% 99.839% ( 1300, 1900 ] 275 0.022% 99.861% ( 1900, 2900 ] 321 0.026% 99.887% ( 2900, 4400 ] 210 0.017% 99.904% ( 4400, 6600 ] 478 0.038% 99.942% ( 6600, 9900 ] 397 0.032% 99.974% ( 9900, 14000 ] 188 0.015% 99.989% ( 14000, 22000 ] 114 0.009% 99.998% ( 22000, 33000 ] 19 0.002% 100.000% ( 33000, 50000 ] 1 0.000% 100.000% ( 50000, 75000 ] 1 0.000% 100.000% STATISTICS: rocksdb.block.cache.miss COUNT : 0 rocksdb.block.cache.hit COUNT : 0 rocksdb.block.cache.add COUNT : 0 rocksdb.block.cache.add.failures COUNT : 0 rocksdb.block.cache.index.miss COUNT : 0 rocksdb.block.cache.index.hit COUNT : 0 rocksdb.block.cache.index.add COUNT : 0 rocksdb.block.cache.index.bytes.insert COUNT : 0 rocksdb.block.cache.filter.miss COUNT : 0 rocksdb.block.cache.filter.hit COUNT : 0 rocksdb.block.cache.filter.add COUNT : 0 rocksdb.block.cache.filter.bytes.insert COUNT : 0 rocksdb.block.cache.data.miss COUNT : 0 rocksdb.block.cache.data.hit COUNT : 0 rocksdb.block.cache.data.add COUNT : 0 rocksdb.block.cache.data.bytes.insert COUNT : 0 rocksdb.block.cache.bytes.read COUNT : 0 rocksdb.block.cache.bytes.write COUNT : 0 rocksdb.bloom.filter.useful COUNT : 20180570 rocksdb.bloom.filter.full.positive COUNT : 1440361 rocksdb.bloom.filter.full.true.positive COUNT : 1243633 rocksdb.persistent.cache.hit COUNT : 0 rocksdb.persistent.cache.miss COUNT : 0 rocksdb.sim.block.cache.hit COUNT : 0 rocksdb.sim.block.cache.miss COUNT : 0 rocksdb.memtable.hit COUNT : 363 rocksdb.memtable.miss COUNT : 1243633 rocksdb.l0.hit COUNT : 50109 rocksdb.l1.hit COUNT : 15334 rocksdb.l2andup.hit COUNT : 1178190 rocksdb.compaction.key.drop.new COUNT : 33391 rocksdb.compaction.key.drop.obsolete COUNT : 0 rocksdb.compaction.key.drop.range_del COUNT : 0 rocksdb.compaction.key.drop.user COUNT : 0 rocksdb.compaction.range_del.drop.obsolete COUNT : 0 rocksdb.compaction.optimized.del.drop.obsolete COUNT : 0 rocksdb.compaction.cancelled COUNT : 0 rocksdb.number.keys.written COUNT : 1311281 rocksdb.number.keys.read COUNT : 1243996 rocksdb.number.keys.updated COUNT : 0 rocksdb.bytes.written COUNT : 1353241992 rocksdb.bytes.read COUNT : 1243996000 rocksdb.number.db.seek COUNT : 0 rocksdb.number.db.next COUNT : 0 rocksdb.number.db.prev COUNT : 0 rocksdb.number.db.seek.found COUNT : 0 rocksdb.number.db.next.found COUNT : 0 rocksdb.number.db.prev.found COUNT : 0 rocksdb.db.iter.bytes.read COUNT : 0 rocksdb.no.file.opens COUNT : 1367 rocksdb.no.file.errors COUNT : 0 rocksdb.stall.micros COUNT : 54373169 rocksdb.db.mutex.wait.micros COUNT : 0 rocksdb.number.multiget.get COUNT : 0 rocksdb.number.multiget.keys.read COUNT : 0 rocksdb.number.multiget.bytes.read COUNT : 0 rocksdb.number.merge.failures COUNT : 0 rocksdb.bloom.filter.prefix.checked COUNT : 0 rocksdb.bloom.filter.prefix.useful COUNT : 0 rocksdb.number.reseeks.iteration COUNT : 0 rocksdb.getupdatessince.calls COUNT : 0 rocksdb.wal.synced COUNT : 0 rocksdb.wal.bytes COUNT : 0 rocksdb.write.self COUNT : 0 rocksdb.write.other COUNT : 0 rocksdb.write.wal COUNT : 0 rocksdb.compact.read.bytes COUNT : 4957501707 rocksdb.compact.write.bytes COUNT : 4927658797 rocksdb.flush.write.bytes COUNT : 1321221256 rocksdb.compact.read.marked.bytes COUNT : 0 rocksdb.compact.read.periodic.bytes COUNT : 0 rocksdb.compact.read.ttl.bytes COUNT : 0 rocksdb.compact.write.marked.bytes COUNT : 0 rocksdb.compact.write.periodic.bytes COUNT : 0 rocksdb.compact.write.ttl.bytes COUNT : 0 rocksdb.number.direct.load.table.properties COUNT : 35 rocksdb.number.superversion_acquires COUNT : 218 rocksdb.number.superversion_releases COUNT : 208 rocksdb.number.superversion_cleanups COUNT : 53 rocksdb.number.block.compressed COUNT : 0 rocksdb.number.block.decompressed COUNT : 0 rocksdb.number.block.not_compressed COUNT : 0 rocksdb.merge.operation.time.nanos COUNT : 0 rocksdb.filter.operation.time.nanos COUNT : 0 rocksdb.row.cache.hit COUNT : 0 rocksdb.row.cache.miss COUNT : 0 rocksdb.read.amp.estimate.useful.bytes COUNT : 0 rocksdb.read.amp.total.read.bytes COUNT : 0 rocksdb.number.rate_limiter.drains COUNT : 0 rocksdb.number.iter.skip COUNT : 0 rocksdb.blobdb.num.put COUNT : 0 rocksdb.blobdb.num.write COUNT : 0 rocksdb.blobdb.num.get COUNT : 0 rocksdb.blobdb.num.multiget COUNT : 0 rocksdb.blobdb.num.seek COUNT : 0 rocksdb.blobdb.num.next COUNT : 0 rocksdb.blobdb.num.prev COUNT : 0 rocksdb.blobdb.num.keys.written COUNT : 0 rocksdb.blobdb.num.keys.read COUNT : 0 rocksdb.blobdb.bytes.written COUNT : 0 rocksdb.blobdb.bytes.read COUNT : 0 rocksdb.blobdb.write.inlined COUNT : 0 rocksdb.blobdb.write.inlined.ttl COUNT : 0 rocksdb.blobdb.write.blob COUNT : 0 rocksdb.blobdb.write.blob.ttl COUNT : 0 rocksdb.blobdb.blob.file.bytes.written COUNT : 0 rocksdb.blobdb.blob.file.bytes.read COUNT : 0 rocksdb.blobdb.blob.file.synced COUNT : 0 rocksdb.blobdb.blob.index.expired.count COUNT : 0 rocksdb.blobdb.blob.index.expired.size COUNT : 0 rocksdb.blobdb.blob.index.evicted.count COUNT : 0 rocksdb.blobdb.blob.index.evicted.size COUNT : 0 rocksdb.blobdb.gc.num.files COUNT : 0 rocksdb.blobdb.gc.num.new.files COUNT : 0 rocksdb.blobdb.gc.failures COUNT : 0 rocksdb.blobdb.gc.num.keys.relocated COUNT : 0 rocksdb.blobdb.gc.bytes.relocated COUNT : 0 rocksdb.blobdb.fifo.num.files.evicted COUNT : 0 rocksdb.blobdb.fifo.num.keys.evicted COUNT : 0 rocksdb.blobdb.fifo.bytes.evicted COUNT : 0 rocksdb.txn.overhead.mutex.prepare COUNT : 0 rocksdb.txn.overhead.mutex.old.commit.map COUNT : 0 rocksdb.txn.overhead.duplicate.key COUNT : 0 rocksdb.txn.overhead.mutex.snapshot COUNT : 0 rocksdb.txn.get.tryagain COUNT : 0 rocksdb.number.multiget.keys.found COUNT : 0 rocksdb.num.iterator.created COUNT : 0 rocksdb.num.iterator.deleted COUNT : 0 rocksdb.block.cache.compression.dict.miss COUNT : 0 rocksdb.block.cache.compression.dict.hit COUNT : 0 rocksdb.block.cache.compression.dict.add COUNT : 0 rocksdb.block.cache.compression.dict.bytes.insert COUNT : 0 rocksdb.block.cache.add.redundant COUNT : 0 rocksdb.block.cache.index.add.redundant COUNT : 0 rocksdb.block.cache.filter.add.redundant COUNT : 0 rocksdb.block.cache.data.add.redundant COUNT : 0 rocksdb.block.cache.compression.dict.add.redundant COUNT : 0 rocksdb.files.marked.trash COUNT : 0 rocksdb.files.deleted.immediately COUNT : 0 rocksdb.error.handler.bg.errro.count COUNT : 0 rocksdb.error.handler.bg.io.errro.count COUNT : 0 rocksdb.error.handler.bg.retryable.io.errro.count COUNT : 0 rocksdb.error.handler.autoresume.count COUNT : 0 rocksdb.error.handler.autoresume.retry.total.count COUNT : 0 rocksdb.error.handler.autoresume.success.count COUNT : 0 rocksdb.memtable.payload.bytes.at.flush COUNT : 1310476288 rocksdb.memtable.garbage.bytes.at.flush COUNT : 430080 rocksdb.secondary.cache.hits COUNT : 0 rocksdb.verify_checksum.read.bytes COUNT : 0 rocksdb.backup.read.bytes COUNT : 0 rocksdb.backup.write.bytes COUNT : 0 rocksdb.remote.compact.read.bytes COUNT : 0 rocksdb.remote.compact.write.bytes COUNT : 0 rocksdb.hot.file.read.bytes COUNT : 0 rocksdb.warm.file.read.bytes COUNT : 0 rocksdb.cold.file.read.bytes COUNT : 0 rocksdb.hot.file.read.count COUNT : 0 rocksdb.warm.file.read.count COUNT : 0 rocksdb.cold.file.read.count COUNT : 0 rocksdb.last.level.read.bytes COUNT : 0 rocksdb.last.level.read.count COUNT : 0 rocksdb.non.last.level.read.bytes COUNT : 10935072025 rocksdb.non.last.level.read.count COUNT : 2496685 rocksdb.block.checksum.compute.count COUNT : 2655561 rocksdb.multiget.coroutine.count COUNT : 0 rocksdb.blobdb.cache.miss COUNT : 0 rocksdb.blobdb.cache.hit COUNT : 0 rocksdb.blobdb.cache.add COUNT : 0 rocksdb.blobdb.cache.add.failures COUNT : 0 rocksdb.blobdb.cache.bytes.read COUNT : 0 rocksdb.blobdb.cache.bytes.write COUNT : 0 rocksdb.read.async.micros COUNT : 0 rocksdb.async.read.error.count COUNT : 0 rocksdb.secondary.cache.filter.hits COUNT : 0 rocksdb.secondary.cache.index.hits COUNT : 0 rocksdb.secondary.cache.data.hits COUNT : 0 rocksdb.table.open.prefetch.tail.miss COUNT : 0 rocksdb.table.open.prefetch.tail.hit COUNT : 0 rocksdb.db.get.micros P50 : 161.803796 P95 : 355.760923 P99 : 552.134723 P100 : 51521.000000 COUNT : 1243996 SUM : 239164462 rocksdb.db.write.micros P50 : 1.726381 P95 : 7.865986 P99 : 2110.547261 P100 : 2460.000000 COUNT : 1311281 SUM : 58197435 rocksdb.compaction.times.micros P50 : 1086666.666667 P95 : 3600000.000000 P99 : 3852208.000000 P100 : 3852208.000000 COUNT : 10 SUM : 13564516 rocksdb.compaction.times.cpu_micros P50 : 973333.333333 P95 : 3600000.000000 P99 : 3818877.000000 P100 : 3818877.000000 COUNT : 10 SUM : 11788489 rocksdb.subcompaction.setup.times.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.sync.micros P50 : 870.000000 P95 : 2900.000000 P99 : 4100.000000 P100 : 4257.000000 COUNT : 20 SUM : 24744 rocksdb.compaction.outfile.sync.micros P50 : 1156.666667 P95 : 7260.000000 P99 : 14278.000000 P100 : 14278.000000 COUNT : 36 SUM : 75020 rocksdb.wal.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.manifest.file.sync.micros P50 : 240.000000 P95 : 1480.000000 P99 : 3890.000000 P100 : 3951.000000 COUNT : 34 SUM : 18419 rocksdb.table.open.io.micros P50 : 5594.339623 P95 : 11603.271605 P99 : 16792.380952 P100 : 21308.000000 COUNT : 1367 SUM : 8152919 rocksdb.db.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.block.compaction.micros P50 : 3.796686 P95 : 168.251355 P99 : 243.335421 P100 : 28437.000000 COUNT : 1201932 SUM : 52725700 rocksdb.read.block.get.micros P50 : 136.228908 P95 : 227.754073 P99 : 392.025519 P100 : 28612.000000 COUNT : 1452251 SUM : 205868698 rocksdb.write.raw.block.micros P50 : 0.608532 P95 : 1.884049 P99 : 3.964898 P100 : 4786.000000 COUNT : 1513650 SUM : 3609478 rocksdb.numfiles.in.singlecompaction P50 : 3.444444 P95 : 17.100000 P99 : 21.020000 P100 : 22.000000 COUNT : 14 SUM : 71 rocksdb.db.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.write.stall P50 : 0.587203 P95 : 2193.496327 P99 : 2200.000000 P100 : 2200.000000 COUNT : 234644 SUM : 54394476 rocksdb.sst.read.micros P50 : 113.872524 P95 : 204.735385 P99 : 342.089562 P100 : 9697.000000 COUNT : 2496721 SUM : 248773299 rocksdb.num.subcompactions.scheduled P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.per.read P50 : 1000.000000 P95 : 1000.000000 P99 : 1000.000000 P100 : 1000.000000 COUNT : 1243996 SUM : 1243996000 rocksdb.bytes.per.write P50 : 1032.000000 P95 : 1032.000000 P99 : 1032.000000 P100 : 1032.000000 COUNT : 1311281 SUM : 1353241992 rocksdb.bytes.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.compressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.bytes.decompressed P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.compression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.decompression.times.nanos P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.read.num.merge_operands P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.key.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.value.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.get.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.multiget.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.seek.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.next.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.prev.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.write.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.read.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.blob.file.sync.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.compression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.blobdb.decompression.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.db.flush.micros P50 : 220000.000000 P95 : 296896.000000 P99 : 296896.000000 P100 : 296896.000000 COUNT : 20 SUM : 4324320 rocksdb.sst.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.index.and.filter.blocks.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.sst.read.per.level P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.error.handler.autoresume.retry.count P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.poll.wait.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.prefetched.bytes.discarded P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 37 SUM : 0 rocksdb.multiget.io.batch.size P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.num.level.read.per.multiget P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.async.prefetch.abort.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0 rocksdb.table.open.prefetch.tail.read.bytes P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0