Fix chopping up multi-byte Unicode characters #21

Merged
prologic merged 1 commits from lyse/feeds:fix-chopping-up-multi-byte-unicode into master 2 weeks ago
lyse commented 2 weeks ago

UTF-8 encoded multi-byte codepoints might have been split in half producing invalid Unicode when truncating to a maximum length. Now we
count runes rather than bytes. This has the side effect, that feeds with multi-byte Unicode characters get larger. But oh well, it should be
actually even better to read.


Unit tests pass. Didn't actually test more than that. 8-)

UTF-8 encoded multi-byte codepoints might have been split in half producing invalid Unicode when truncating to a maximum length. Now we count runes rather than bytes. This has the side effect, that feeds with multi-byte Unicode characters get larger. But oh well, it should be actually even better to read. ----- Unit tests pass. Didn't actually test more than that. 8-)
lyse added 1 commit 2 weeks ago
d988d00320 Fix chopping up multi-byte Unicode characters
prologic merged commit 12c271ea14 into master 2 weeks ago
lyse deleted branch fix-chopping-up-multi-byte-unicode 2 weeks ago
continuous-integration/drone/pr Build is passing
The pull request has been merged as 12c271ea14.
Sign in to join this conversation.
No reviewers
No Milestone
No Assignees
1 Participants
Notifications
Due Date

No due date set.

Dependencies

This pull request currently doesn't have any dependencies.

Loading…
There is no content yet.