Use tempfile instead of memory to prepare the plaintext and cipher bu… #145
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ffers This greatly reduces the memory footprint when creating the pkg. Example: Peak memory consumption when creating the PKG for RE2 goes from ~3.7G to ~1.2G. Signed-off-by: Ronnie Sahlberg <ronniesahlberg@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…ffers
During pack, we read all the files into buffers, then copy them into a plaintext buffer and then encrypt to a cipher buffer.
Resulting in all the file data being stored three times in memory.
This patch moves processing of the crypto to be file based instead of memory buffer based and eliminates two of the three copies being stored in memory.
This greatly reduces the memory footprint when creating the pkg. Example: Peak memory consumption when creating the PKG for a PSX Classics of ResidentEvil2 goes from ~3.7G to ~1.2G.