HBASE-29761: The HBase UI's Debug Dump is not redacting sensitive information #7545

kgeisz · 2025-12-13T00:37:45Z

https://issues.apache.org/jira/browse/HBASE-29761

This pull request fixes an issue where the HBase UI's Debug Dump feature was still showing sensitive config information (such as passwords). Consider an hbase-site.xml file that contains the following:

  <property>
    <name>hbase.zookeeper.property.ssl.trustStore.password</name>
    <value>kevin-zk-pw</value>
  </property>
  <property>
    <name>ssl.client.truststore.password</name>
    <value>kevin-ssl-truststore-pw</value>
  </property>
  <property>
    <name>hbase.rpc.tls.truststore.password</name>
    <value>kevin-tls-truststore-pw</value>
  </property>
  <property>
    <name>ssl.server.keystore.password</name>
    <value>kevin-ssl-keystore-pw</value>
  </property>
  <property>
    <name>hadoop.security.sensitive-config-keys</name>
    <value>
      secret$
      password$
      hbase\.zookeeper\.property\.ssl\.trustStore\.password$
      ssl.keystore.pass$
      fs.s3a.server-side-encryption.key
      fs.s3a.*.server-side-encryption.key
      fs.s3a.encryption.algorithm
      fs.s3a.encryption.key
      fs.s3a.secret.key
      fs.s3a.*.secret.key
      fs.s3a.session.key
      fs.s3a.*.session.key
      fs.s3a.session.token
      fs.s3a.*.session.token
      fs.azure.account.key.*
      fs.azure.oauth2.*
      fs.adl.oauth2.*
      fs.gs.encryption.*
      fs.gs.proxy.*
      fs.gs.auth.*
      credential$
      oauth.*secret
      oauth.*password
      oauth.*token
      hadoop.security.sensitive-config-keys
    </value>
  </property>

Here, hadoop.security.sensitive-config-keys specifies various regexes for what config property names should have their values redacted. However, before this change, properties such as the ones listed above (hbase.zookeeper.property.ssl.trustStore.password, etc.) would still have their sensitive contents present in plain text in the HBase UI's Debug Dump.

With this change, these sensitive values are now redacted and replaced with ******. The issue was occurring because the wrong Configuration.writeXml() method was being called in MasterDumpServlet and RSDumpServlet. Before, the method being used resulted in a call chain of Configuration.writeXml() methods that eventually led to ConfigRedactor being null. This change directly calls the writeXml() method that was at the end of the call chain, which allows the ConfigRedactor to be established.

In addition, the unit tests created in this PR reused methods found in TestMasterStatusPage.java and TestRSStatusPage.java. These common methods have been moved to a new TestServerHttpUtils.java to eliminate repeated code.

…ormation Change-Id: I7f0cf9f096727272764252d8e7f6b8c6f5fc91c0

kgeisz · 2025-12-13T01:01:34Z

Hi @PDavid, can you please review this PR? Also, I am still having trouble getting the HBase UI to work locally, so I have not been able to manually test this change when I branch off of the latest version of master.

PDavid · 2025-12-13T12:07:10Z

Hi @PDavid, can you please review this PR? Also, I am still having trouble getting the HBase UI to work locally, so I have not been able to manually test this change when I branch off of the latest version of master.

Hi @kgeisz, many thanks for fixing this. I pulled your branch, built it locally and tested the debug dump of both Master and RegionServer Status servlet and the fix is working. 👍

However the hadoopcheck failed with Hadoop v3.3.6:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project hbase-server: Compilation failure: Compilation failure: 
[ERROR] /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-7545/yetus-general-check/src/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/http/RSDumpServlet.java:[94,11] no suitable method found for writeXml(<nulltype>,java.io.OutputStreamWriter,org.apache.hadoop.conf.Configuration)
[ERROR]     method org.apache.hadoop.conf.Configuration.writeXml(java.io.OutputStream) is not applicable
[ERROR]       (actual and formal argument lists differ in length)
[ERROR]     method org.apache.hadoop.conf.Configuration.writeXml(java.io.Writer) is not applicable
[ERROR]       (actual and formal argument lists differ in length)
[ERROR]     method org.apache.hadoop.conf.Configuration.writeXml(java.lang.String,java.io.Writer) is not applicable
[ERROR]       (actual and formal argument lists differ in length)
[ERROR] /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-7545/yetus-general-check/src/hbase-server/src/main/java/org/apache/hadoop/hbase/master/http/MasterDumpServlet.java:[94,11] no suitable method found for writeXml(<nulltype>,java.io.OutputStreamWriter,org.apache.hadoop.conf.Configuration)
[ERROR]     method org.apache.hadoop.conf.Configuration.writeXml(java.io.OutputStream) is not applicable
[ERROR]       (actual and formal argument lists differ in length)
[ERROR]     method org.apache.hadoop.conf.Configuration.writeXml(java.io.Writer) is not applicable
[ERROR]       (actual and formal argument lists differ in length)
[ERROR]     method org.apache.hadoop.conf.Configuration.writeXml(java.lang.String,java.io.Writer) is not applicable
[ERROR]       (actual and formal argument lists differ in length)

Can you please check it?

PDavid · 2025-12-13T12:13:41Z

hbase-server/src/test/java/org/apache/hadoop/hbase/util/TestServerHttpUtils.java

+import java.net.URL;
+import org.apache.hadoop.hbase.LocalHBaseCluster;
+
+public class TestServerHttpUtils {


Thanks for extracting these methods to a reusable utility class. 👍

There is a related checkstyle error:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7545/2/artifact/yetus-general-check/output/results-checkstyle-hbase-server.txt

I guess it is about we need to add a private constructor as this class only has static methods.

stoty · 2025-12-13T14:57:24Z

hbase-server/src/main/java/org/apache/hadoop/hbase/master/http/MasterDumpServlet.java

      Configuration conf = master.getConfiguration();
      out.flush();
-      conf.writeXml(os);
+      conf.writeXml(null, os, conf);


Overloading the same method to have very different security-related behaviour is iffy, but there is nothing HBase can do about that.

stoty

+1 LGTM

Kota-SH

lgtm

Change-Id: I961ff80833c4da6f879cfeafa2620ecfaa0e7e84

kgeisz · 2025-12-16T01:01:45Z

Hi @PDavid, it turned out there is a compatibility issue with Hadoop versions older than Hadoop 3.4. YARN-11308 introduced the new Configuration.writeXml() method that I was using in my original change. Since this was causing a build error, and also would not be compatible with branch-2, I decided to manually perform the redaction myself.

I introduced a new method to StateDumpServlet.java that performs the redaction. I also decided to have the redacted text be <redacted> rather than ****** (I can change it back if needed). I also added a private constructor to TestServerHttpUtils.java, which will hopefully stop the spotless error.

kgeisz · 2025-12-16T01:02:24Z

cc. @taklwu

Apache-HBase · 2025-12-16T01:44:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	5m 25s		master passed
+1 💚	compile	3m 38s		master passed
+1 💚	checkstyle	1m 6s		master passed
+1 💚	spotbugs	1m 50s		master passed
+1 💚	spotless	0m 51s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 10s		the patch passed
+1 💚	compile	3m 28s		the patch passed
+1 💚	javac	3m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 0s	/results-checkstyle-hbase-server.txt	hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	spotbugs	1m 51s		the patch passed
+1 💚	hadoopcheck	11m 43s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚	spotless	0m 46s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 11s		The patch does not generate ASF License warnings.
		43m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7545/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7545
JIRA Issue	HBASE-29761
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 2fe49208d070 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `dcb2185`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7545/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-12-16T05:17:12Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	5m 9s		master passed
+1 💚	compile	1m 28s		master passed
+1 💚	javadoc	0m 42s		master passed
+1 💚	shadedjars	9m 5s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	4m 24s		the patch passed
+1 💚	compile	1m 6s		the patch passed
+1 💚	javac	1m 6s		the patch passed
+1 💚	javadoc	0m 30s		the patch passed
+1 💚	shadedjars	6m 27s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	221m 20s		hbase-server in the patch passed.
		255m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7545/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7545
JIRA Issue	HBASE-29761
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 4b14a63054a6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `dcb2185`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7545/3/testReport/
Max. process+thread count	5191 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7545/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

PDavid · 2025-12-16T07:06:10Z

Hi @PDavid, it turned out there is a compatibility issue with Hadoop versions older than Hadoop 3.4. YARN-11308 introduced the new Configuration.writeXml() method that I was using in my original change. Since this was causing a build error, and also would not be compatible with branch-2, I decided to manually perform the redaction myself.

I introduced a new method to StateDumpServlet.java that performs the redaction. I also decided to have the redacted text be <redacted> rather than ****** (I can change it back if needed). I also added a private constructor to TestServerHttpUtils.java, which will hopefully stop the spotless error.

Many thanks for looking into this @kgeisz and I think it is a good idea to do the redaction ourselves. 👍

I'd not introduce another XML element instead of the text content we have now because if any application parses the output of the dump servlet would need to be adjusted after this change - this would break compatibility. So I'd keep the "*****" instead.
What do you all think?

HBASE-29761: The HBase UI's Debug Dump is not redacting sensitive inf…

3a45a84

…ormation Change-Id: I7f0cf9f096727272764252d8e7f6b8c6f5fc91c0

kgeisz force-pushed the HBASE-29761-hbase-ui-debug-dump-showing-passwords branch from b67f4bc to 3a45a84 Compare December 13, 2025 00:43

This comment has been minimized.

Sign in to view

PDavid reviewed Dec 13, 2025

View reviewed changes

stoty reviewed Dec 13, 2025

View reviewed changes

stoty approved these changes Dec 13, 2025

View reviewed changes

NihalJain approved these changes Dec 15, 2025

View reviewed changes

Kota-SH approved these changes Dec 15, 2025

View reviewed changes

Manually perform redaction to ensure backward compatibility

dcb2185

Change-Id: I961ff80833c4da6f879cfeafa2620ecfaa0e7e84

HBASE-29761: The HBase UI's Debug Dump is not redacting sensitive information #7545

Are you sure you want to change the base?

HBASE-29761: The HBase UI's Debug Dump is not redacting sensitive information #7545

Conversation

kgeisz commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kgeisz commented Dec 13, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

PDavid commented Dec 13, 2025

Uh oh!

PDavid Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

stoty Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

stoty left a comment

Choose a reason for hiding this comment

Uh oh!

Kota-SH left a comment

Choose a reason for hiding this comment

Uh oh!

kgeisz commented Dec 16, 2025

Uh oh!

kgeisz commented Dec 16, 2025

Uh oh!

Apache-HBase commented Dec 16, 2025

Uh oh!

Apache-HBase commented Dec 16, 2025

Uh oh!

PDavid commented Dec 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

kgeisz commented Dec 13, 2025 •

edited

Loading

PDavid commented Dec 16, 2025 •

edited

Loading