Unify nRF & ESP BLE interface

[liquidraver]

Core Stuff

• Unified common code into SerialBLECommon.h base class
  (it should reside in a helpers/bluetooth folder with all the nrf/esp files, but this PR is not for tidying up)

• ESP32: Migrated from ESP32 BLE library to NimBLE - Re-pair could be needed! Full erase would be preferable for updating to clear out old BT library stuff from the flash. Factory reset does not clear that.

  • this also resolves xiao c3 bootloops because of too much sensors (-~500k from flash)

Connection

• Reject second connection attempts (single connection only) (edge case!)
• Improved disconnect handling with reason codes and initiator tracking (for debugging who is to blame for disconnects)

Dynamic connection parameters

• Added sync mode: switches to low-latency params (0 latency, 12-24ms interval) for high throughput
• Sync mode when the fw auto-detects large frame activity (40+ bytes, 3+ frames in 1.5s window)
• Falls back to default mode (3 latency, 12-36ms interval) after 5s inactivity
• Tracks connection parameter updates via BLE events

Queue & Transfer Improvements

• Circular queue implementation replaces linear array shifts - less RAM/CPU
• Added send interval throttling (8ms minimum) - undetectable by user, but helps stabilize the ble stack, on ESP this is mandatory!
• Improved retry logic with throttle window (250ms) - throttle sending when we detect spotty BT connection

ESP32 Specific:
Well... everything that nRF had already are now in ESP's interface :) even "turbo mode"

The two platform's BLE interface is now more or less the same (with library dependent differences of course)

[fdlamotte]

well did a first test with thinknode M5 ... it connects but I have no response to APPSTART :(

I have an heltec_v3, I'll try with it ...

update: I erased the device, removed it from the PC and did the pairing again ... now I can connect

Is it necessary to to a full erase before using it with the new lib ?

[liquidraver]

well did a first test with thinknode M5 ... it connects but I have no response to APPSTART :(

I have an heltec_v3, I'll try with it ...

update: I erased the device, removed it from the PC and did the pairing again ... now I can connect

Is it necessary to to a full erase before using it with the new lib ?

On my test devices it was not necessary, but the MAC is the same and bluetooth keys on ESP are on the NVS partition, they can conflict. A factory reset won't erase that.

On paper IDF and nimBLE does not use the same namespace on NVS, but it can cause hard-to-debug bugs.
(we should add NVS erase to our factory reset function, so it will mean the same as on nRF, i.e.: completely clean slate)

Completely ditching ESP-IDF and using a brand new lib so starting fresh on flash is recommended.

[ripplebiz]

Nice work!

[recrof]

regarding NVS clearing: #1294

[liquidraver]

regarding NVS clearing: #1294

this won't be enough now because after upgrade if the user can't reconnect, then can't invoke factory reset.

we can create code to call nvs format exactly once, but after the countless nRF corruption errors from the past weeks(months?) I would highlight in the next release's changelog that a backup - full erase - restore is highly recommended.
then there is no need to code in a one-time nvs clear and we can resolve lingering nRF corruption errors too

need to re-think the BLE bonding invalidation, and how to make this a seamless update for users.

[liquidraver]

we can do a one-time NVS clear, but we can't ask the phone to re-pair. iOS notoriusly disregards anything and just tries to connect forcefully to a bond.