-
Notifications
You must be signed in to change notification settings - Fork 2
Fix masking for fill_value and missing_value for numpy 2.4.0 #304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #304 +/- ##
==========================================
+ Coverage 86.04% 86.21% +0.17%
==========================================
Files 7 7
Lines 645 653 +8
==========================================
+ Hits 555 563 +8
Misses 90 90 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
activestorage/storage.py
Outdated
| if missing_value is not None: | ||
| data = np.ma.masked_equal(data, missing_value) | ||
| if isinstance(missing_value, np.ndarray) or isinstance(missing_value, list): | ||
| data = np.ma.masked_where(data == missing_value, data) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Surely this only works when missing_value broadcasts to data, which in general it wont.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
well I'm running out of any valid numpy function that can mask with an array then 😁
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry, I got distracted by the news, how do you even apply a missing_value mask array if it's not broadcastable? I am really not familiar with anything missing value that's dim > 1 (or not scalar) - please suggest code, buds 🍺
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@davidhassell sorry - only just now that I refactored - given this is for storage.py, and that's just an emulator of Reductionist, I decided to pop a ValueError instead of going through with your albeit solid suggestion to go through the individual missing values - you reckon that'll do it? 🍺
Description
I merged too fast #303 (like a .303 bullet hehe) but @davidhassell mentioned it's not complete since missing_value could be a multi-dim array, not just a size one, as I thought. This should fix that.
Reminder: numpy 2.4.0 doesn't allow stuff like ma.masked_equal or other such funcs that set fill_values, to have arrays passed for fill_value, eg their deprecation:
This implementation here should be backwards compatible.
Before you get started
Checklist