If every file is just a number, how can lossless compression work at all?
Here is the argument that is bothering me. Any file is a string of bits, so any file is a number. A lossless compressor maps each input to an output that is supposed to be shorter, and the decompressor maps it back, so the mapping must be reversible.
But there are fewer short bit strings than long ones. If every input of a given length maps to a strictly shorter output, then several inputs must share an output, and then decompression cannot be unambiguous.
So compression appears to be impossible, and yet it obviously works. Where is the flaw in my reasoning?
@entropy_elif · last wk.
There is no flaw. Your argument is correct and it is a real theorem: no lossless compressor can shrink every input. For any compressor, some inputs must get longer.
The counting argument you gave is exactly the proof, and it is worth appreciating that you derived it yourself.
What you have not noticed is that this does not conflict with compression working, because real compressors do not claim to shrink everything. They shrink some inputs — a very particular subset — and expand the rest slightly.
And the subset they shrink is the one we care about. Text, images, audio, source code, logs, executables: all deeply patterned, all a vanishingly small fraction of the space of possible bit strings. Almost all bit strings are noise, almost none of them are ever stored in a file, and a compressor is a bet that its input comes from the patterned corner rather than from the enormous random middle.
So compression is not magic, it is exploitation of a bias in which files actually exist.
Reply
Report