Skip to content

HDDS-14225. [DO NOT MERGE] Upgrade RocksDB from 7.7.3 to 10.4.2#9813

Draft
smengcl wants to merge 3 commits intoapache:masterfrom
smengcl:rocksdb-v10-upgrade
Draft

HDDS-14225. [DO NOT MERGE] Upgrade RocksDB from 7.7.3 to 10.4.2#9813
smengcl wants to merge 3 commits intoapache:masterfrom
smengcl:rocksdb-v10-upgrade

Conversation

@smengcl
Copy link
Contributor

@smengcl smengcl commented Feb 24, 2026

Generated-by: GPT-5.3-Codex

What changes were proposed in this pull request?

Bump RocksDB version from 7.7.3 to 10.4.2. While maintaining compatibility and not breaking anything.

  1. Since RocksDB 9, BlockBasedTableOptions.format_version=6 is the default. Files written in format_version 6 cannot be read by RocksDB < 8.6.0. This PR explicitly sets the default to version 5 to be compatible with 7.7.3 in the case where Ozone gets downgraded before it gets finalized after an upgrade. ref: https://github.com/facebook/rocksdb/releases/tag/v9.0.0
  2. DO NOT MERGE until v10.9.0 or higher RocksDB JNI is available. Potential forward compatibility bug < v10.9.0: https://github.com/facebook/rocksdb/releases/tag/v10.9.1

Fix a bug where compaction with range deletion can persist kTypeMaxValid in MANIFEST as file metadata. kTypeMaxValid is not supposed to be persisted and can change as new value types are introduced. This can cause a forward compatibility issue where older versions of RocksDB don't recognize kTypeMaxValid from newer versions. A new placeholder value type kTypeTruncatedRangeDeletionSentinel is also introduced to replace kTypeMaxValid when reading existing SST files' metadata from MANIFEST. This allows us to strengthen some checks to avoid using kTypeMaxValid in the future.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14225

How was this patch tested?

  • Most RocksDB related tests are fixed.
  • Pending full CI.

@smengcl smengcl changed the title HDDS-14225. Upgrade RocksDB from 7.7.3 to 10.4.2 HDDS-14225. [DO NOT MERGE] Upgrade RocksDB from 7.7.3 to 10.4.2 Feb 24, 2026
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @smengcl for working on this.

We should detect OS at runtime, not build time. Single build (on any OS) should create binaries for all supported operating systems.

  1. dependencyManagement in root POM should include an entry for rocksdbjni artifact with each supported classifier, as well as the non-classified artifact:

      <dependency>
        <groupId>org.rocksdb</groupId>
        <artifactId>rocksdbjni</artifactId>
        <version>${rocksdb.version}</version>
      </dependency>
      <dependency>
        <groupId>org.rocksdb</groupId>
        <artifactId>rocksdbjni</artifactId>
        <version>${rocksdb.version}</version>
        <classifier>linux64</classifier>
      </dependency>
      <dependency>
        <groupId>org.rocksdb</groupId>
        <artifactId>rocksdbjni</artifactId>
        <version>${rocksdb.version}</version>
        <classifier>osx</classifier>
      </dependency>
      ...
  2. hdds-rocks-native should unpack all of these

  3. other modules should continue depending on platform-independent rocksdbjni (no classifier)

Thus changes in most pom.xml files are not needed, nor is rocksdbjni.classifier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants