Transformer LM

Sun, 11 Jun 2023 00:00:00 +0000

Inside the Architecture of Game Repacks

Thu, 02 Feb 2023 00:00:00 +0000

If you have ever spent time in PC gaming communities, you have almost certainly encountered repacks. Within hours of a game’s release, a highly specialized subset of software engineers known in the digital underground as repackers will release a version of that exact same game, compressed to a fraction of its original size. A huge 80GB chunk reduced to a 20GB download, all while remaining fully functional upon installation.

To the average user, seeing a game cut to a quarter of its original download size looks like black magic. Standard file compression utilities like 7-Zip, WinRAR, or Windows ZIP usually struggle to trim off even 5% from a modern game folder.

How do repackers achieve these extreme ratios?

This phenomenon is not achieved through simple file zipping. Standard archiving tools like WinRAR or 7-Zip won’t be able to achieve anything similar or even come close for a task like this, often yielding negligible space savings when applied to pre-compiled game data. Instead, game repacking is a highly complex, multi-stage discipline that sits at the intersection of reverse engineering, advanced data compression mathematics, and systems architecture.

To understand how repackers manage to shrink the huge size of modern software, we need to look at what a repack is really made of, from dismantling a game’s proprietary file structure to the practical (and theoretical) limits of dictionary-based compression.

Modern AAA Bloat

Before you can shrink a game, you have to understand why it’s so big to begin with. Modern video games aren’t just code. They’re massive collections of multimedia data. The executable that holds the game’s real logic usually stays under a few hundred megabytes, and the remaining bulk—about 99% of the disk space is mostly made up of assets.

High-Fidelity Textures and Mipmapping
As screens have moved from 1080p to 4K (and higher), texture files have ballooned in size. A single 4K texture, if it’s stored without compression, can take up tens of megabytes. On top of that, many engines use mipmaps smaller, pre-made versions of the same texture that the game swaps in when objects are far away. Mipmaps make rendering faster and reduce visual shimmering, but they also add roughly about 33% more storage for the texture package.
Uncompressed and Localized Audio
Audio is one of the biggest space hogs. To avoid extra processing during gameplay, games often ship sound in uncompressed PCM-style formats. If the game has a layered soundtrack, thousands of sound effects, and fully voiced dialogue, the audio banks can swell quickly. Then comes localization: if the game releases in multiple languages and each one includes full voice acting, the install size grows almost linearly with the number of languages. So even if one language is what you use, you may end up downloading a huge amount of useless data.
Data Duplication to Speed Up Loading
Older game builds were designed around mechanical HDDs, where reading scattered data is slow because the drive head has to physically seek. To keep loading smooth, developers sometimes duplicated the same assets many times across different archive sections things like common textures or repeated objects so each level’s required data could be laid out more sequentially on disk. SSDs (especially NVMe) made this less necessary, but some older pipelines and content packaging approaches can still leave behind duplication.

Breakdown of the Virtual File System

Before a repacker can shrink a game, it has to be willing to “break” the way the game is packaged. It can’t just run compression on the whole install folder, because the engine usually stores assets inside proprietary containers (like Unreal’s .pak, Frostbite’s .cas, or Unity’s .assets).

These containers act like a Virtual File System: they’re built for fast access during gameplay, not for maximum storage efficiency. Inside, the data is often already lightly compressed for speed (for example with LZ4 or Oodle Kraken) and may also be encrypted.

That’s where repack compression gets tricky: common general-purpose compressors depend on finding repeated byte patterns. If the data is encrypted or already compressed well, it looks almost random high entropy so it doesn’t compress any further. To get big savings, the repacker has to get past the engine’s container format and extract the underlying raw assets.

Reverse Engineering with QuickBMS

QuickBMS is a tool for extracting/repacking data from packed or proprietary archive formats, using custom scripts that describe how to locate file tables, offsets, encryption/compression, and naming. To reduce all that, repackers lean on QuickBMS. Those scripts decode the container structure by identifying file headers, offset tables, and the internal layout the engine uses. Once the repacker understands that structure, it extracts every raw asset textures, audio chunks, models, scripts out of the container and into a huge folder of mostly uncompressed files.

Now that the repacker has individual building blocks, it can use compression methods that aim for a lopsided outcome: heavy compute time during packing, but reasonable decompression time for the end user.

Solid Archiving

Repacking usually relies on solid archives. In a normal ZIP, each file is compressed separately. In a solid archive, many files are stitched into one continuous stream and compressed together.

That matters for games because lots of assets are similar. Think of thousands of XML files with recurring structures, or many UI textures that share patterns. When the compressor can see across file boundaries, it can reuse more repeated sequences, boosting the compression ratio.

The Dictionary Size

Modern repacks often use dictionary-based compression (especially LZMA2-style variants or Zstandard). These work by learning a dictionary of recurring byte sequences. When the same sequence shows up again, the compressor replaces it with a reference instead of storing the bytes directly.

How well this works depends heavily on dictionary size. A dictionary of 1GB means the decompressor can look back up to that far to find matching sequences bigger dictionary, more chances to reuse patterns, but also higher memory and overhead.

Patching and Deduplication

Before algorithmic compression, repackers often run custom deduplication scripts. If the extracted files contain 50 identical copies of a 10MB texture (due to the HDD optimization mentioned earlier), the repacker will delete 49 of them. They then write a custom script instructing the installer to dynamically recreate the missing 49 copies upon installation using hardlinks or direct byte-copying.

Lossless vs. Lossy

Compression algorithms alone cannot shrink 80GB to 20GB. To achieve the most staggering size reductions, repackers must alter the heaviest assets: audio and video. This introduces the distinction between Lossless and Lossy repacks.

Lossless Repack

A lossless repack means that when the game is installed, the files are mathematically identical (1:1 hash match) to the original game files. No audio quality is downgraded; no video pixels are lost.
To achieve this with audio, repackers exploit the inefficiency of game audio formats. Game engines often use ADPCM or raw PCM WAV files wrapped in middleware containers (like Wwise .pck files).

Lossy Repack

In a lossy repack,the actual quality of the media is permanently altered.
4K Bink (.bik) cutscenes with bitrates of 50 Mbps are aggressively re-encoded down to 1080p or 720p at 5 Mbps using HEVC (H.265). The repacker utilizes tools like RAD Video Tools to convert the standard MP4/MKV back into the engine’s proprietary .bik format so the game engine can still read them. This alone can take off 20GB off a modern game. Uncompressed audio is heavily downsampled (e.g., from 48kHz to 32kHz) or re-encoded to lossy formats like opus or ogg vorbis, drastically reducing the size of the audio banks.
While the game is fully playable, a lossy repack sacrifices audiovisual fidelity for extreme bandwidth savings.

Final Decompression

The trade-off for all this clever compression is time and stress on your hardware. Repackers push work onto installation: instead of making the download smaller for free, they shift the heavy lifting to your CPU, RAM, and disk I/O.

Installing a highly compressed repack can be a brutal workload—often driving many-core CPUs to 100% usage for hours.

What happens during decompression

When you run setup.exe, the installer’s unpacking engine typically runs a staged pipeline:

Memory Allocation: The installer maps the dictionary into the system RAM.
CPU Decoding: The CPU processes the LZMA/Zstd data streams, resolving dictionary pointers into raw bytes.
Disk I/O: The decompressed raw bytes are written to a temporary folder on the storage drive.
Recompilation: Custom scripts run in the background, taking the raw bytes and repacking them back into the game engine’s proprietary .pak or .cas formats.
Verification: The installer runs a CRC (Cyclic Redundancy Check) or MD5 hash check on the final reconstructed files to ensure they perfectly match the retail release.

The architecture of a video game repack is a masterclass in data science. Reverse engineering proprietary formats, extracting redundancy, and then using aggressive heuristics plus brute-force compute to squeeze data down as far as possible.

Even though repacks are tied to piracy, the underlying engineering problem is genuinely impressive from a technical perspective: instead of optimizing for runtime performance, repackers optimize for storage density by dismantling engine containers and extracting redundancy that general-purpose packaging leaves behind.

initial-test-page

Thu, 29 Dec 2022 00:00:00 +0000

“If the enemy is in range, so are you”

-Unknown

Home on pwn-e4