DeflateDecompressor: fix inflater memory leak#1137
Open
pjfanning wants to merge 4 commits into
Open
Conversation
Motivation: DeflateDecompressor allocates an Inflater per stream but never explicitly releases its native memory. Under GCs with relaxed off-heap reclamation (e.g. ZGC), repeated request decompression can accumulate unreclaimed native buffers and eventually OOM. Modification: - Add createInflater(noWrap) factory method to DeflateDecompressor to allow test injection of a custom Inflater - Track the current inflater in a var within createLogic - Add idempotent cleanupInflater() that calls inflater.end() - Override postStop() to call cleanupInflater(), covering normal completion, failure, and cancellation - Add focused tests in DeflateSpec verifying the inflater is released on successful decode, early cancellation, and truncation Result: The Inflater's native memory is reclaimed promptly when the stage terminates, regardless of termination path. Tests: - Not run - sbt/builds skipped per user request to conserve credits References: Refs apache#1134
He-Pin
requested changes
Jul 6, 2026
He-Pin
left a comment
He-Pin
left a comment
Member
There was a problem hiding this comment.
Thanks for the fix. I found two things that should be fixed before merge: one is the Scala 2.13 compile failure, and the other is a lifecycle hole when the parser loops back for another deflate stream.
He-Pin
approved these changes
Jul 6, 2026
The Sink.ignore future completes when take(1) sends Complete downstream, but the Cancel signal that triggers postStop() (and thus end()) is a separate actor message dispatched afterwards. This means awaitResult can return before end() has been called, yielding endCalls = 0. Add a CountDownLatch to TrackingInflater that is counted down inside end(), and call inflater.awaitEnd() after awaitResult() in the cancellation test to eliminate the race.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation:
DeflateDecompressor allocates an Inflater per stream but never explicitly releases its native memory. Under GCs with relaxed off-heap reclamation (e.g. ZGC), repeated request decompression can accumulate unreclaimed native buffers and eventually OOM.
Modification:
Result:
The Inflater's native memory is reclaimed promptly when the stage terminates, regardless of termination path.
References:
Refs #1134 and #1133