Aug 2026 draft
why PDF compression is harder than it looks
my first idea was "just compress the bytes harder". building pdfctl taught me a PDF is a container of already-compressed parts, and that changes everything.
a PDF is not one document
a PDF file is a bag of numbered objects: content streams that draw text, image XObjects, fonts, metadata, and a cross-reference table at the end recording the byte offset of every object. most streams already carry a filter declaring how they are encoded. text and vector content is usually FlateDecode (zlib). photos are usually DCTDecode (JPEG) or JPXDecode (JPEG 2000). fonts are embedded subsets, themselves compressed.
so "the file" is really a dozen different compression domains glued together, each already squeezed with an algorithm suited to its data. there is no single byte stream for a compressor to find patterns across. that was my first wrong mental model, and everything confused me until i dropped it.
the already-compressed problem
the naive move is running deflate over the whole file again. on a photo-heavy PDF this gains roughly nothing, typically under 1 percent, because JPEG output is already near-random bytes and a second compression pass cannot find structure in it. I once watched a "compressed" output come out larger than the input, since the new stream wrappers and dictionaries cost more bytes than the recompression saved.
the real wins live elsewhere. uncompressed content streams (some producers emit them raw) compress dramatically, often 5 to 10 times smaller under flate. duplicated objects are common: the same logo image embedded separately on all 40 pages instead of referenced once, or a full font embedded when only 30 glyphs are used. removing one duplicate 200 KB image beats any compression-level tuning on the remaining streams.
offsets fight back
the mechanical reason compression work is fiddly: the xref table stores exact byte offsets, so shrinking any object shifts every byte after it and invalidates the table. you cannot edit a PDF in place the way you edit text. every serious operation rewrites the file and rebuilds the offsets, which is why pdfctl works on a temp file and atomically moves it into place.
worse, the spec allows incremental updates: instead of rewriting, a writer appends changed objects plus a new xref section at the end. do that ten times and the file carries ten generations of dead objects. i have seen files where the dead generations were a third of the total size. "compressing" such a file without consolidating generations is polishing the live third while ignoring the dead two thirds.
what surprised me
what surprised me is the ranking of wins. from biggest to smallest in the files i tested: deduplicating repeated images and fonts, consolidating incremental generations, compressing uncompressed streams, subsetting fonts, and only then, dead last, turning up the deflate level. my starting instinct had the list almost exactly backwards. compression level is the knob everyone reaches for first and it matters least, because the format already applied the right codec per part at write time.
what i would do differently: parse before touching anything. dump every object with its filter, decoded size, and reference count first, then decide. the one time i skipped that and went straight to recompressing streams, i spent an afternoon optimizing objects that turned out to be referenced zero times. dead objects compress beautifully and it does not matter at all.
all notes