I am reading the paper Automatic Static Unpacking of Malware Binaries (Kevin Coogan et al.) with the goal of trying to reproduce the given experimental results (with
tElock, etc), and studying how it can be expanded for other cases.
As far as I understand, the authors first use pointer analysis to extract the unpacking code by detecting transition points (i.e. the point separates the execution of the normal instructions from the runtime generated instructions), then the unpacker code is extracted using the backward slicing analysis from this point.
After a process of "code punning" and "reassembly" (e.g. patch out the defense code, fix some relocation problems, etc.), one could obtain a new binary whose each unpacked block is initially marked as a
s-object (abbr. for section objects). Then the new binary can be emulated (if I understood correctly) where
s-object(s) will be filled out by unpacked code.
While I might be able to imagine some limits of static analysis applied in the paper (e.g. inaccuracies of backward static slicing, pointer analysis, side effects...), and the implicit hypothesis about the existence of the transition points, I still cannot figure out how the described static unpacker works.
First, the authors say that the backward static slicing is applied since the context of the problem is unstructured binaries (and that is true since malicious codes are unstructured), but
Problem 1: how can the value-set analysis be applied?
since we have no hope to restore abstract locations (used by value-set analysis) in unstructured binaries. For example, in the following unpacking stub of
mov edx, 0x135 mov ebx, 0x401000 mov eax 0x6bf00803 unpack: sub [ebx], eax nop sub eax, 0x15e3c0 add ebx, 0x4 dec ecx jne unpack jmp _oep _oep: ...
I suppose that there would exist no abstract locations, no?
Second, the authors say that each
s-object contains meta-data (i.e. name, size, ...) about some section, as cited in
V.B.3.1 of the paper:
an s-object... contains meta-data about the section it presents... These meta-data are obtained from section table of the binary...
Problem 2: how can we be sure that the unpacked code must be fit in a section (whose info can be obtained by parsing the binary header)?
That would be probably true in the case of "pure"
UPX where the unpacked codes are located in the section
UPX0, but this is not true in general (e.g. the case above of
Since there is a step of address translation described in
V.B.3.1 which arise from the difference between the normal runtime unpacking process of the binary and the unpacking process of the static unpacker, I assume that the output of this unpacker is a new binary contains only unpacked code and no unpacking stub. But
Problem 3: how can it deal with multiple level packed programs?
For example, the experimental results given in the
Figure 5 of the paper has the case of
Peed-44 which uses a customized
UPX contains at least
2 unpacking levels: which form should be considered as the "real" unpacked code?
So my question is
Is my understanding about the paper correct? (that should be not) then where did I misunderstand?