Compress large text changes in ChangeHistory #720
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| package com.philkes.notallyx.data.model | ||
|
|
||
| @JvmInline | ||
| value class BodyString(val value: String) { | ||
| override fun toString(): String = value | ||
|
|
||
| fun isEmpty(): Boolean = value.isEmpty() | ||
|
|
||
| fun isNotEmpty(): Boolean = value.isNotEmpty() | ||
|
|
||
| fun contains(other: String, ignoreCase: Boolean = false): Boolean = | ||
| value.contains(other, ignoreCase) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,89 @@ | ||||||
| package com.philkes.notallyx.utils | ||||||
|
|
||||||
| import android.provider.ContactsContract.CommonDataKinds.StructuredName.PREFIX | ||||||
|
There was a problem hiding this comment. Remove unused import. Line 3 imports Apply this diff: -import android.provider.ContactsContract.CommonDataKinds.StructuredName.PREFIX📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| import android.util.Base64 | ||||||
| import android.util.Log | ||||||
| import com.github.luben.zstd.Zstd | ||||||
| import com.philkes.notallyx.data.model.Converters | ||||||
| import com.philkes.notallyx.data.model.SpanRepresentation | ||||||
| import com.philkes.notallyx.utils.CompressUtility.COMPRESSION_THRESHOLD | ||||||
| import java.io.ByteArrayInputStream | ||||||
| import java.io.ByteArrayOutputStream | ||||||
| import java.util.zip.GZIPInputStream | ||||||
| import java.util.zip.GZIPOutputStream | ||||||
| import org.json.JSONObject | ||||||
|
|
||||||
| /** | ||||||
| * Shared compression utilities for large text payloads to decrease memory and storage usage. | ||||||
| * - For EditText state (text + spans), we store a GZIP-compressed JSON as ByteArray. | ||||||
|
There was a problem hiding this comment. Correct the misleading comment. The comment states "GZIP-compressed JSON as ByteArray" but the implementation uses Zstd compression (line 37), not GZIP. Apply this diff: - * - For EditText state (text + spans), we store a GZIP-compressed JSON as ByteArray.
+ * - For EditText state (text + spans), we store a Zstd-compressed JSON as ByteArray.📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| * - For BaseNote.body (String persisted in DB), we store GZIP(Base64) with a small prefix marker. | ||||||
| */ | ||||||
| object CompressUtility { | ||||||
|
|
||||||
| // Threshold in characters for when to compress text (approximately 7KB) | ||||||
| const val COMPRESSION_THRESHOLD: Int = 7_000 | ||||||
|
|
||||||
| // Prefix to mark a String as compressed (so we can store in a TEXT column). | ||||||
| private const val PREFIX: String = "GZ:" | ||||||
|
|
||||||
| // region Text + Spans (ByteArray) | ||||||
|
|
||||||
| /** Compresses text and spans using GZIP compression into a ByteArray. */ | ||||||
| fun compressTextAndSpans(text: String, spans: List<SpanRepresentation>): ByteArray { | ||||||
| val jsonObject = JSONObject() | ||||||
| jsonObject.put("text", text) | ||||||
| jsonObject.put("spans", Converters.spansToJSONArray(spans)) | ||||||
| val bytes = jsonObject.toString().toByteArray(Charsets.UTF_8) | ||||||
| return Zstd.compress(bytes, 4) | ||||||
| } | ||||||
|
|
||||||
| /** Decompresses text and spans that were compressed with GZIP. */ | ||||||
| fun decompressTextAndSpans(compressedData: ByteArray): Pair<String, List<SpanRepresentation>> { | ||||||
| val decompressedSize = Zstd.decompressedSize(compressedData) | ||||||
| val result = ByteArray(decompressedSize.toInt()) | ||||||
| Zstd.decompress(result, compressedData) | ||||||
| val jsonString = result.toString(Charsets.UTF_8) | ||||||
| // val bis = ByteArrayInputStream(compressedData) | ||||||
| // val jsonString = GZIPInputStream(bis).use { gzipIS -> | ||||||
| // gzipIS.readBytes().toString(Charsets.UTF_8) | ||||||
| // } | ||||||
| val jsonObject = JSONObject(jsonString) | ||||||
| val text = jsonObject.getString("text") | ||||||
| val spansArray = jsonObject.getJSONArray("spans") | ||||||
| val spans = Converters.jsonToSpans(spansArray) | ||||||
| return Pair(text, spans) | ||||||
| } | ||||||
|
|
||||||
| // endregion | ||||||
|
|
||||||
| // region String-only (BaseNote.body) | ||||||
|
|
||||||
| /** Compress String if above threshold. Returns original if already small. */ | ||||||
| fun compressIfNeeded(text: String): String { | ||||||
| if (text.length <= COMPRESSION_THRESHOLD) return text | ||||||
| return try { | ||||||
| val bos = ByteArrayOutputStream() | ||||||
| GZIPOutputStream(bos).use { it.write(text.toByteArray(Charsets.UTF_8)) } | ||||||
| val b64 = Base64.encodeToString(bos.toByteArray(), Base64.NO_WRAP) | ||||||
| PREFIX + b64 | ||||||
| } catch (e: Exception) { | ||||||
| Log.w("CompressUtility", "Failed to compress, returning original", e) | ||||||
| text | ||||||
| } | ||||||
| } | ||||||
|
|
||||||
| /** Decompress String if it was previously compressed with [compressIfNeeded]. */ | ||||||
| fun decompressIfNeeded(text: String): String { | ||||||
| if (!text.startsWith(PREFIX)) return text | ||||||
| val b64 = text.removePrefix(PREFIX) | ||||||
| return try { | ||||||
| val bytes = Base64.decode(b64, Base64.NO_WRAP) | ||||||
| val bis = ByteArrayInputStream(bytes) | ||||||
| GZIPInputStream(bis).use { it.readBytes().toString(Charsets.UTF_8) } | ||||||
| } catch (e: Exception) { | ||||||
| Log.w("CompressUtility", "Failed to decompress, returning original", e) | ||||||
| text | ||||||
| } | ||||||
| } | ||||||
| // endregion | ||||||
| } | ||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Undo cursor position regressed
When the user moves the caret (no text change) and then types again, we now reuse the previous change’s
newValue, sostateBeforekeeps the old selection instead of the currentselectionStart. Undoing the new edit jumps the cursor back to the prior position, breaking the expected undo behaviour.Please rebuild
stateBeforewith the current selection while still reusing the cached text snapshot. For example:This keeps the clone optimisation but ensures undo restores the caret to the position the user actually had before the change.
🤖 Prompt for AI Agents