VED-1006 Improve delta error handling

[dlzhry2nhs]

Summary

• Routine Change

Minor improvement to how we handle errors in the Delta Lambda and hopefully improves the readability too.

Key changes:

• removed SNS Topic that was setup for Lambda failures. This was not being used (no one was subscribed to it) and it was inconsistent with the behaviour within the function which was to deliver the full Kinesis event to SQS where failures would be securely stored.
• refactored so instead of sending the full Kinesis event, even when some records succeeded and some failed, we just send the ones which failed (i.e. critical DB error, data was not persisted etc.) to SQS so they can be investigated and retried.
• removed the try - catch wrapper. As the process_records method is fully wrapped (intentionally, it seems) the only thing it was catching was a malformed event, which can only really happen if you're messing around or AWS release something catastrophic. In such cases just let it fail -> get the full error -> also forwarded to SQS. We will get a notification from the alarm anyway.

Reviews Required

• Dev

Review Checklist

ℹ️ This section is to be filled in by the reviewer.

• I have reviewed the changes in this PR and they fill all of the acceptance criteria of the ticket.
• If there were infrastructure, operational, or build changes, I have made sure there is sufficient evidence that the changes will work.
• If there were changes that are outside of the regular release processes e.g. account infrastructure to setup, manual setup for external API integrations, secrets to set, then I have checked that the developer has flagged this to the Tech Lead as release steps.
• I have checked that no Personal Identifiable Data (PID) is logged as part of the changes.

[github-actions]

This branch is working on a ticket in the NHS England VED JIRA Project. Here's a handy link to the ticket:

VED-1006

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud