ref(service): Add metadata API, fix delete orphans, simplify BigTable backend

[jan-auer]

Add get_metadata to the Backend trait so that HEAD requests, tombstone checks, and delete operations can avoid full payload reads. BigTable uses ColumnQualifierRegexFilter for column-level reads; GCS skips alt=media.

Tombstone-safe delete sequence

When deleting an object that lives on the long-term backend, the high-volume backend holds a redirect tombstone. The delete sequence ensures the tombstone is only removed after the long-term object is gone, so data is never orphaned:

1. delete_non_tombstone(high_volume)
   → if NOT a tombstone: row deleted, done
   → if tombstone: row left intact, returns Tombstone

2. delete_object(long_term)        — remove the real data
3. delete_object(high_volume)      — clean up the tombstone

BigTable conditional delete

BigTable implements delete_non_tombstone as a single CheckAndMutateRow RPC. A regex predicate on the metadata column detects is_redirect_tombstone:true:

• Predicate matches (tombstone): no mutations — the row stays intact.
• Predicate does not match (regular object): DeleteFromRow — the row is removed.

This keeps the common-case delete at 1 RPC while preserving the tombstone for the two-phase long-term cleanup.

Additional changes

• TTI bumping for metadata-only reads: All backends now bump TTI on get_metadata calls. BigTable conditionally fetches the payload column and rewrites the row only when a bump is needed (~1/day due to debounce), keeping the common case at 1 RPC. The bump is best-effort — failures don't fail the read.
• BigTable backend simplification: Extracted shared helpers (RowData, read_row, with_retry) that eliminate duplicated retry loops, cell parsing, and expiry checking.

[cursor]

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

[matt-codecov]

insane edge case but somebody could store custom metadata that contains this text. if we can't deserialize metadata in these filters then this field name should include characters that we don't allow in custom metadata keys/values or we should put our name in the string or something

[jan-auer]

Excellent catch!

A better way would be to use protobuf instead of JSON because we can then read fields inside the column directly - I believe this is even possible for predicates. However, that would require a migration at this point and in the meanwhile we need a solution for this anyway. So we'll stick with JSON and try to harden it.

[jan-auer]

I addressed this now with a somewhat dirty workaround: We're locking in the tombstone marker at first position, so we can match the metadata at the start.