Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 32 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,19 +69,46 @@ Pre-built instrumentations: `prometheus-metrics-instrumentation-jvm`, `-caffeine
## Code Style

- **Formatter**: Google Java Format (enforced via Spotless)
- **Line length**: 100 characters
- **Line length**: 100 characters (enforced for ALL files including Markdown, Java, YAML, etc.)
- **Indentation**: 2 spaces
- **Static analysis**: Error Prone with NullAway (`io.prometheus.metrics` package)
- **Logger naming**: Logger fields must be named `logger` (not `log`, `LOG`, or `LOGGER`)
- **Assertions in tests**: Use static imports from AssertJ (`import static org.assertj.core.api.Assertions.assertThat`)
- **Empty catch blocks**: Use `ignored` as the exception variable name
- **Markdown code blocks**: Always specify language (e.g., ` ```java`, ` ```bash`, ` ```text`)

## Linting and Validation

- **IMPORTANT**: Always run `mise run build` after modifying Java files to ensure all lints, code formatting (Spotless), static analysis (Error Prone), and checkstyle checks pass
- **IMPORTANT**: Always run `mise run lint:super-linter` after modifying non-Java files (YAML, Markdown, shell scripts, JSON, etc.)
- Super-linter is configured to only show ERROR-level messages via `LOG_LEVEL=ERROR` in `.github/super-linter.env`
- Local super-linter version is pinned to match CI (see `.mise/tasks/lint/super-linter.sh`)
**CRITICAL**: These checks MUST be run before creating any commits. CI will fail if these checks fail.

### Java Files

- **ALWAYS** run `mise run build` after modifying Java files to ensure:
- Code formatting (Spotless with Google Java Format)
- Static analysis (Error Prone with NullAway)
- Checkstyle validation
- Build succeeds (tests are skipped; run `mise run test` or `mise run test-all` to execute tests)

### Non-Java Files (Markdown, YAML, JSON, shell scripts, etc.)

- **ALWAYS** run `mise run lint:super-linter` after modifying non-Java files
- Super-linter will **auto-fix** many issues (formatting, trailing whitespace, etc.)
- It only reports ERROR-level issues (configured via `LOG_LEVEL=ERROR` in `.github/super-linter.env`)
- Common issues caught:
- Lines exceeding 100 characters in Markdown files
- Missing language tags in fenced code blocks
- Table formatting issues
- YAML/JSON syntax errors

### Running Linters

```bash
# After modifying Java files (run BEFORE committing)
mise run build

# After modifying non-Java files (run BEFORE committing)
mise run lint:super-linter
```

## Testing

Expand Down
88 changes: 88 additions & 0 deletions docs/content/getting-started/metric-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,94 @@ for [Histogram.Builder](/client_java/api/io/prometheus/metrics/core/metrics/Hist
for a complete list of options. Some options can be configured at runtime,
see [config]({{< relref "../config/config.md" >}}).

### Custom Bucket Boundaries

The default bucket boundaries are designed for measuring request durations in seconds. For other
use cases, you may want to define custom bucket boundaries. The histogram builder provides three
methods for this:

**1. Arbitrary Custom Boundaries**

Use `classicUpperBounds(...)` to specify arbitrary bucket boundaries:

```java
Histogram responseSize = Histogram.builder()
.name("http_response_size_bytes")
.help("HTTP response size in bytes")
.classicUpperBounds(100, 1000, 10000, 100000, 1000000) // bytes
.register();
```

**2. Linear Boundaries**

Use `classicLinearUpperBounds(start, width, count)` for equal-width buckets:

```java
Histogram queueSize = Histogram.builder()
.name("queue_size")
.help("Number of items in queue")
.classicLinearUpperBounds(10, 10, 10) // 10, 20, 30, ..., 100
.register();
```

**3. Exponential Boundaries**

Use `classicExponentialUpperBounds(start, factor, count)` for exponential growth:

```java
Histogram dataSize = Histogram.builder()
.name("data_size_bytes")
.help("Data size in bytes")
.classicExponentialUpperBounds(100, 10, 5) // 100, 1k, 10k, 100k, 1M
.register();
```

### Native Histograms with Custom Buckets (NHCB)

Prometheus supports a special mode called Native Histograms with Custom Buckets (NHCB) that uses
schema -53. In this mode, custom bucket boundaries from classic histograms are preserved when
converting to native histograms.

The Java client library automatically supports NHCB:

1. By default, histograms maintain both classic (with custom buckets) and native representations
2. The classic representation with custom buckets is exposed to Prometheus
3. Prometheus servers can convert these to NHCB upon ingestion when configured with the
`convert_classic_histograms_to_nhcb` scrape option

Example:

```java
// This histogram will work seamlessly with NHCB
Histogram apiLatency = Histogram.builder()
.name("api_request_duration_seconds")
.help("API request duration")
.classicUpperBounds(0.01, 0.05, 0.1, 0.5, 1.0, 5.0, 10.0) // custom boundaries
.register();
```

On the Prometheus side, configure the scrape job:

```yaml
scrape_configs:
- job_name: "my-app"
scrape_protocols: ["PrometheusProto"]
convert_classic_histograms_to_nhcb: true
static_configs:
- targets: ["localhost:9400"]
```

{{< hint type=note >}}
NHCB is useful when:

- You need precise bucket boundaries for your specific use case
- You're migrating from classic histograms and want to preserve bucket boundaries
- Exponential bucketing from standard native histograms isn't a good fit for your distribution
{{< /hint >}}

See [examples/example-custom-buckets](https://github.com/prometheus/client_java/tree/main/examples/example-custom-buckets) <!-- editorconfig-checker-disable-line -->
for a complete example with Prometheus and Grafana.

Histograms and summaries are both used for observing distributions. Therefore, the both implement
the `DistributionDataPoint` interface. Using the `DistributionDataPoint` interface directly gives
you the option to switch between histograms and summaries later with minimal code changes.
Expand Down
170 changes: 170 additions & 0 deletions examples/example-custom-buckets/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,170 @@
# Native Histograms with Custom Buckets (NHCB) Example

This example demonstrates how to use native histograms with custom bucket boundaries (NHCB) in
Prometheus Java client. It shows three different types of custom bucket configurations and how
Prometheus converts them to native histograms with schema -53.

## What are Native Histograms with Custom Buckets?

Native Histograms with Custom Buckets (NHCB) is a Prometheus feature that combines the benefits of:

- **Custom bucket boundaries**: Precisely defined buckets optimized for your specific use case
- **Native histograms**: Efficient storage and querying capabilities of native histograms

When you configure Prometheus with `convert_classic_histograms_to_nhcb: true`, it converts classic
histograms with custom buckets into native histograms using schema -53, preserving the custom
bucket boundaries.

## Example Metrics

This example application generates three different histogram metrics demonstrating different
bucket configuration strategies:

### 1. API Latency - Arbitrary Custom Boundaries

```java
Histogram apiLatency = Histogram.builder()
.name("api_request_duration_seconds")
.classicUpperBounds(0.01, 0.05, 0.1, 0.5, 1.0, 5.0, 10.0)
.register();
```

**Use case**: Optimized for typical API response times in seconds.

### 2. Queue Size - Linear Boundaries

```java
Histogram queueSize = Histogram.builder()
.name("message_queue_size")
.classicLinearUpperBounds(10, 10, 10) // 10, 20, 30, ..., 100
.register();
```

**Use case**: Equal-width buckets for monitoring queue depth or other discrete values.

### 3. Response Size - Exponential Boundaries

```java
Histogram responseSize = Histogram.builder()
.name("http_response_size_bytes")
.classicExponentialUpperBounds(100, 10, 6) // 100, 1k, 10k, 100k, 1M, 10M
.register();
```

**Use case**: Data spanning multiple orders of magnitude (bytes, milliseconds, etc).

## Build

This example is built as part of the `client_java` project:

```shell
./mvnw package
```

This creates `./examples/example-custom-buckets/target/example-custom-buckets.jar`.

## Run

With the JAR file present, run:

```shell
cd ./examples/example-custom-buckets/
docker-compose up
```

This starts three Docker containers:

- **[http://localhost:9400/metrics](http://localhost:9400/metrics)** - Example application
- **[http://localhost:9090](http://localhost:9090)** - Prometheus server (with NHCB enabled)
- **[http://localhost:3000](http://localhost:3000)** - Grafana (user: _admin_, password: _admin_)

You might need to replace `localhost` with `host.docker.internal` on macOS or Windows.

## Verify NHCB Conversion

### 1. Check Prometheus Configuration

The Prometheus configuration enables NHCB conversion:

```yaml
scrape_configs:
- job_name: "custom-buckets-demo"
scrape_protocols: ["PrometheusProto"]
convert_classic_histograms_to_nhcb: true
scrape_classic_histograms: true
```

### 2. Verify in Prometheus

Visit [http://localhost:9090](http://localhost:9090) and run queries:

```promql
# View histogram metadata (should show schema -53 for NHCB)
prometheus_tsdb_head_series

# Calculate quantiles from custom buckets
histogram_quantile(0.95, rate(api_request_duration_seconds[1m]))

# View raw histogram structure
api_request_duration_seconds
```

### 3. View in Grafana

The Grafana dashboard at [http://localhost:3000](http://localhost:3000) shows:

- p95 and p50 latencies for API endpoints (arbitrary custom buckets)
- Queue size distribution (linear buckets)
- Response size distribution (exponential buckets)

## Key Observations

1. **Custom Buckets Preserved**: The custom bucket boundaries you define are preserved when
converted to NHCB (schema -53).

2. **Dual Representation**: By default, histograms maintain both classic and native
representations, allowing gradual migration.

3. **Efficient Storage**: Native histograms provide more efficient storage than classic histograms
while preserving your custom bucket boundaries.

4. **Flexible Bucket Strategies**: You can choose arbitrary, linear, or exponential buckets based
on your specific monitoring needs.

## When to Use Custom Buckets

Consider using custom buckets (and NHCB) when:

- **Precise boundaries needed**: You know the expected distribution and want specific bucket edges
- **Migrating from classic histograms**: You want to preserve existing bucket boundaries
- **Specific use cases**: Default exponential bucketing doesn't fit your distribution well
- Temperature ranges (might include negative values)
- Queue depths (discrete values with linear growth)
- File sizes (exponential growth but with specific thresholds)
- API latencies (specific SLA boundaries)

## Differences from Standard Native Histograms

| Feature | Standard Native Histograms | NHCB (Schema -53) |
| ----------------- | ------------------------------- | --------------------------------- |
| Bucket boundaries | Exponential (base 2^(2^-scale)) | Custom boundaries |
| Use case | General-purpose | Specific distributions |
| Mergeability | Can merge with same schema | Cannot merge different boundaries |
| Configuration | Schema level (0-8) | Explicit boundary list |

## Cleanup

Stop the containers:

```shell
docker-compose down
```

## Further Reading

<!-- editorconfig-checker-disable -->
<!-- markdownlint-disable MD013 -->

- [Prometheus Native Histograms Specification](https://prometheus.io/docs/specs/native_histograms/)
- [Prometheus Java Client Documentation](https://prometheus.github.io/client_java/)
- [OpenTelemetry Exponential Histograms](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#exponentialhistogram)
26 changes: 26 additions & 0 deletions examples/example-custom-buckets/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
version: "3"
services:
example-application:
image: eclipse-temurin:25.0.1_8-jre@sha256:9d1d3068b16f2c4127be238ca06439012ff14a8fdf38f8f62472160f9058464a
network_mode: host
volumes:
- ./target/example-custom-buckets.jar:/example-custom-buckets.jar
command:
- /opt/java/openjdk/bin/java
- -jar
- /example-custom-buckets.jar
prometheus:
image: prom/prometheus:v3.9.1@sha256:1f0f50f06acaceb0f5670d2c8a658a599affe7b0d8e78b898c1035653849a702
network_mode: host
volumes:
- ./docker-compose/prometheus.yml:/prometheus.yml
command:
- --enable-feature=native-histograms
- --config.file=/prometheus.yml
grafana:
image: grafana/grafana:12.3.2@sha256:ba93c9d192e58b23e064c7f501d453426ccf4a85065bf25b705ab1e98602bfb1
network_mode: host
volumes:
- ./docker-compose/grafana-datasources.yaml:/etc/grafana/provisioning/datasources/grafana-datasources.yaml
- ./docker-compose/grafana-dashboards.yaml:/etc/grafana/provisioning/dashboards/grafana-dashboards.yaml
- ./docker-compose/grafana-dashboard-custom-buckets.json:/etc/grafana/grafana-dashboard-custom-buckets.json
Loading
Loading