[Orbit] Add min_delta parameter to NewBestMetric #13594
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a min_delta parameter to NewBestMetric. Motivation: In many training scenarios, we want to avoid saving checkpoints or triggering actions for negligible improvements (e.g., floating point noise). min_delta allows users to specify a minimum threshold for improvement. Changes:
Added min_delta to NewBestMetric.init.
Updated test() logic to require new_value > best_value + min_delta (if higher is better).
Added unit test test_new_best_metric_with_min_delta in new_best_metric_test.py.
Type of change
For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.
Note: Please delete options that are not relevant.
Tests
I added a new test case
test_new_best_metric_with_min_deltainorbit/actions/new_best_metric_test.pyto verify the new functionality.Test Configuration:
python -m orbit.actions.new_best_metric_testChecklist