- What versions/modifications of UPX exist?
- How do they differ? What features do they have?
I will ignore that there's multiple compression algorithms in UPX and that there's been multiple versions of UPX.
Generally when people ask if it's plain or vanilla UPX it's because malware and other software likes to take UPX and modify it slightly so that it can't be unpacked with the standard UPX executable and so that anti viruses will have a harder time unpacking it. It's not very effective at counteracting reverse engineering.
First, let's see UPX structure.
CMP / JNZ for DLLs parameter checks
Pushad, set registers
optional NOP alignment
- whether it's NRV or LZMA
- UPX transform relative calls and jumps into absolute ones, to improve compression.
- load libraries, resolve APIs
Reset section flags
- clean stack
- jump to the original EntryPoint
For more details, here is a commented IDA (free version) IDB of a UPX-ed PE.
modified UPX variants
Simple parts like prologue/epilogue are easy to modify, and are consequently often modified:
- basic polymorphism: replacing an instruction with an equivalent
- moving them around with jumps
Complex parts like decompression, calls restoration, imports loading are usually kept unmodified, so usually, custom code is inserted between them:
- an anti-debug
- an extra xor loop (after decompression)
- a marker that will be checked further in the unpacked code, so that the file knows it was unpacked.
As the prologue doesn't do much, it's also trivial to copy it to the EntryPoint of a non UPX-packed PE, to fool identifiers and fake UPX packing.
I'm not sure if this is what you're asking, but UPX has multiple ways of compressing a given format. For example, an ELF - can be decompressed directly into memory - can be decompressed into /tmp and executed from there
By default the first option is preferred, but I don't think it's mandatory. See the UPX Manual for details.