use ByteString toArrayUnsafe in new deflater code#1135
Draft
pjfanning wants to merge 5 commits into
Draft
Conversation
pjfanning
changed the title
Member
Author
|
@mkurz does this look ok? |
He-Pin
reviewed
Jul 6, 2026
| try { | ||
| val input = if (appendTail) data ++ EmptyStoredBlock else data | ||
| inflater.setInput(input.toArray) | ||
| inflater.setInput(input.toArrayUnsafe()) |
Member
There was a problem hiding this comment.
can we use the asInputStream here
Member
Author
There was a problem hiding this comment.
would need something like new InflaterInputStream(byteString.asInputStream)
Member
Author
There was a problem hiding this comment.
actually it's CRC32 that is what uses the bytes from the ByteString not the Inflater directly
He-Pin
approved these changes
Jul 6, 2026
pjfanning
commented
Jul 6, 2026
| deflater.setInput(data.toArray) | ||
| val output = new ByteArrayOutputStream() | ||
| deflater.setInput(data.toArrayUnsafe()) | ||
| val output = new ByteArrayOutputStream(1024) |
Member
Author
There was a problem hiding this comment.
ByteArrayOutputStream starts with 32 bytes by default and you can end up allocating multiple new arrays while you grow the data. Starting with a reasonable value can help.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Code looks like it should be ok to use toArrayUnsafe, that no other code should be able to modify the array data.
toArrayUnsafe() on ByteString is only really helpful for ByteStrings directly backed by 1 byte array with no offset. Otherwise, it gives you the same as toArray.
I don't know if the costs outweigh the benefits but if used an InflaterInputStream, we can get an InputStream from a ByteString that can avoid array copies for all ByteString types - but with extra expense to manage the stream, etc.
new InflaterInputStream(byteString.asInputStream)