Reliable algorithm to extract overlay of a PE

Question

I write a Portable Executable (PE) library that also provides finding the starting offset of the overlay (appended data to the PE that is not mapped into memory).

My algorithm finding the overlay offset looks like this so far:

public long getOverlayOffset() throws IOException {
        if (offset == null) {
            SectionTable table = data.getSectionTable();
            offset = 0L;
            for (SectionTableEntry section : table.getSectionEntries()) {
                long pointerToRaw = section.get(POINTER_TO_RAW_DATA);
                long sizeOfRaw = section.get(SIZE_OF_RAW_DATA);
                long virtSize = section.get(VIRTUAL_SIZE);
                //see https://code.google.com/p/corkami/wiki/PE#section_table: "if bigger than virtual size, then virtual size is taken. "
                //and: "a section can have a null VirtualSize: in this case, only the SizeOfRawData is taken into consideration. "
                if(virtSize != 0 && sizeOfRaw > virtSize) { 
                    sizeOfRaw = virtSize;
                }
                long endPoint = pointerToRaw + sizeOfRaw;
                if (offset < endPoint) {
                    offset = endPoint;
                }
            }
        }
        if(offset > file.length()) {
            offset = file.length();
        }
        return offset;
    }

I used corkami as a source to get to know some of the odds in calculating the overlay offset. I do not only want it to be robust, but also accurate. Did I miss something? What else do I have to put into consideration?

Note: There was a similar question here: How can one extract the appended data of a Portable Executable? But it doesn't cover a reliable algorithm so far. As I understand it, using a tool suffices in that question.

Answer 1

You appear to be missing the corner cases such as mis-aligned pointers (should round down) and sizes (should round up).

However, even the rounding has corner cases - the physical pointer should be rounded down to a multiple of 512, regardless of the value in the header, but the read size is rounded up by using a combination of the file alignment and 4kb. The virtual size is always rounded up to a multiple of 4kb, regardless of the value in the header.

Digital signatures must be overlays. This is a check that is enforced now by Windows, for security reasons. If not an overlay, the file won't load.

You need something like this (and filealign comes from the PE header):

        long pointerToRaw = section.get(POINTER_TO_RAW_DATA);
        long alignedpointerToRaw = pointerToRaw & ~0x1ff;
        long sizeOfRaw = section.get(SIZE_OF_RAW_DATA);
        long readsize = ((pointerToRaw + sizeOfRaw) + filealign - 1) & ~(filealign - 1)) - alignedpointerToRaw;
        readsize = min(readsize, (sizeOfRaw + 0xfff) & ~0xfff);
        long virtsize = section.get(VIRTUAL_SIZE);

        if (virtsize)
        {
            readsize = min(readsize, (virtsize + 0xfff) & ~0xfff);
        }

Then you have "alignedpointerToRaw" as the starting position, and "readsize" as the number of bytes in the section. Sum these to find the end of the section. You need to perform this calculation for all sections (because physical data might not be sequential in the file). The largest sum is the end of the image. Anything beyond that is overlay.

Answer 2

For the sake of completeness: This is is the code I made based on Peter Ferrie's suggestions

/**
 * Calculates the beginning of the overlay
 * 
 * @return the file offset to the beginning of overlay
 */
public long getOverlayOffset() {
    SectionTable table = data.getSectionTable();
    OptionalHeader opt = data.getOptionalHeader();
    offset = 0L;
    List<SectionHeader> headers = table.getSectionHeaders();
    if(headers.size() == 0) {
        offset = file.length(); //offset for sectionless PE's
    }
    for (SectionTableEntry section : table.getSectionEntries()) {
        long alignedPointerToRaw = section.get(POINTER_TO_RAW_DATA)
                & ~0x1ff;
        long endPoint = getReadSize(section) + alignedPointerToRaw;
        if (offset < endPoint) {
            offset = endPoint;
        }
    }
    if (offset > file.length()) {
        offset = file.length();
    }
    return offset;
}

/**
 * Determines the the number of bytes that is read for the section.
 * 
 * @param section
 * @return section bytes that are read
 */
private long getReadSize(SectionTableEntry section) {
    long pointerToRaw = section.get(POINTER_TO_RAW_DATA);
    long virtSize = section.get(VIRTUAL_SIZE);
    long sizeOfRaw = section.get(SIZE_OF_RAW_DATA);
    long fileAlign = data.getOptionalHeader().get(
            WindowsEntryKey.FILE_ALIGNMENT);
    long alignedPointerToRaw = section.get(POINTER_TO_RAW_DATA) & ~0x1ff;
    long readSize = alignedUp(pointerToRaw + sizeOfRaw, fileAlign)
            - alignedPointerToRaw;
    readSize = Math.min(readSize,
            alignedUp(section.get(SIZE_OF_RAW_DATA), 0x1000));
    if (virtSize != 0) {
        readSize = Math.min(readSize,
                alignedUp(section.get(VIRTUAL_SIZE), 0x1000));
    }
    return readSize;
}

/**
 * Returns the value rounded up to a multiple of alignTo.
 * 
 * @param value
 * @param alignTo
 * @return value rounded up to a multiple of alignTo
 */
private long alignedUp(long value, long alignTo) {
    if (value % alignTo != 0) {
        value = (value + alignTo - 1) & ~(alignTo - 1);
    }
    return value;
}

Answer 3

Have you considered using Microsoft's code to do it? Reverse the loader just enough to find the pointer that you're looking for once the image is loaded, stop it from executing any actual code beyond parsing the PE. The offset to your desired pointer in ntdll or wherever will be version-specific but you'll get a super-accurate parser.