ti_threatconnect: update pipeline to lowercase all hashes#17455
ti_threatconnect: update pipeline to lowercase all hashes#17455GShepherdTC wants to merge 2 commits intoelastic:mainfrom
Conversation
ReviewersBuildkite won't run for external contributors automatically; you need to add a comment:
NOTE: https://github.com/elastic/integrations/blob/main/.buildkite/pull-requests.json contains all those details. |
efd6
changed the title
efd6
left a comment
efd6
left a comment
There was a problem hiding this comment.
Suggested commit message:
ti_threatconnect: lowercase all hash values in indicator processing
Hash values should be normalized to lowercase for consistent matching
and deduplication across threat intelligence sources.
though this probably needs expansion. I think we want to know what visualisation and saved search impacts this will have.
| # newer versions go on top | ||
| - version: "2.0.0" | ||
| changes: | ||
| - description: Update all hash types to be lowercase |
There was a problem hiding this comment.
| - description: Update all hash types to be lowercase | |
| - description: Normalize all hash fields to be lowercase. |
andrewkroh
left a comment
andrewkroh
left a comment
There was a problem hiding this comment.
I opened a ECS issue to advise using lowercase for the hashes.
While reviewing I noticed that ECS threat.indicator.file.size and threat.indicator.geo.location were present in the TC data but completely missing from the ECS mapping. Can you please map those as well to improve ECS coverage?
| copy_from: threat_connect.indicator.md5 | ||
| ignore_empty_value: true | ||
| if: ctx.threat_connect?.indicator?.type instanceof String && ctx.threat_connect.indicator.type.contains('File') | ||
| - lowercase: |
There was a problem hiding this comment.
This needs to occur before the value is copied into the ECS threat.indicator.file.hash.md5 field.
Please check the sha1 and sha256 get the same treatment if they need it.
andrewkroh
added
Integration:ti_threatconnect
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
4 participants
Proposed commit message
Updated pipeline to lowercase all hashes
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Run the pipeline test. I updated the response data to match.
Related issues
Screenshots
No changes to UI.