HBASE-29889 Add XXH3 Hash Support to Bloom Filter using hash4j#7782
HBASE-29889 Add XXH3 Hash Support to Bloom Filter using hash4j#7782jinhyukify wants to merge 6 commits intoapache:masterfrom
Conversation
| <!-- hash --> | ||
| <hash4j.version>0.29.0</hash4j.version> |
There was a problem hiding this comment.
Not fully sure if I should add this dependency here
hbase-shaded/pom.xml
Outdated
| <filter> | ||
| <!-- We are not ready to use JDK 25 yet, and these classes cause shading | ||
| errors on older ASM versions. | ||
| Safe to remove if upgrading the shade plugin to 3.6.1 or later. | ||
| --> | ||
| <artifact>com.dynatrace.hash4j:hash4j</artifact> | ||
| <excludes> | ||
| <exclude>META-INF/versions/25/**</exclude> | ||
| </excludes> | ||
| </filter> |
There was a problem hiding this comment.
fyi: we get the following errors without this
.m2/repository/com/dynatrace/hash4j/hash4j/0.29.0/hash4j-0.29.0.jar entry META-INF/versions/25/com/dynatrace/hash4j/hashing/FFMUtil$MemorySegmentByteAccess.class:
java.lang.IllegalArgumentException: Unsupported class file major version 69
There was a problem hiding this comment.
Another possible fix is upgrading the maven-shade-plugin from 3.6.0 to 3.6.1, since ASM starts supporting Java 25 in 3.6.1.
However, we’re not ready to adopt this jdk version.
There was a problem hiding this comment.
We can bump the shade plugin version, altough we only test on JDK17 official, as far as I know lots of our users have already run HBase with JDK21, so maybe there are users want to run HBase with JDK25.
| @Override | ||
| public <T> long hash64(HashKey<T> hashKey, long seed) { | ||
| Hasher64 hasher = seed == 0L ? Hashing.xxh3_64() : Hashing.xxh3_64(seed); | ||
| return hasher.hashBytesToLong(hashKey, 0, hashKey.length(), HashKeyByteAccess.INSTANCE); |
There was a problem hiding this comment.
I think tests for the hash function itself are outside the scope of this change.
The actual verification was done separately, and all hash functions behave as expected.
The test did on jinhyukify@ed1c821
|
CI Failures on large-wave-1, large-wave-3
All tests pass on my local. |
2 participants
Jira https://issues.apache.org/jira/browse/HBASE-29889
This is the version that implements XXH3 using the hash4j library.
Please refer to the comment above for more details.
#7740 (comment)