7

I write a Portable Executable (PE) library that also provides finding the starting offset of the overlay (appended data to the PE that is not mapped into memory).

My algorithm finding the overlay offset looks like this so far:

public long getOverlayOffset() throws IOException {
        if (offset == null) {
            SectionTable table = data.getSectionTable();
            offset = 0L;
            for (SectionTableEntry section : table.getSectionEntries()) {
                long pointerToRaw = section.get(POINTER_TO_RAW_DATA);
                long sizeOfRaw = section.get(SIZE_OF_RAW_DATA);
                long virtSize = section.get(VIRTUAL_SIZE);
                //see https://code.google.com/p/corkami/wiki/PE#section_table: "if bigger than virtual size, then virtual size is taken. "
                //and: "a section can have a null VirtualSize: in this case, only the SizeOfRawData is taken into consideration. "
                if(virtSize != 0 && sizeOfRaw > virtSize) { 
                    sizeOfRaw = virtSize;
                }
                long endPoint = pointerToRaw + sizeOfRaw;
                if (offset < endPoint) {
                    offset = endPoint;
                }
            }
        }
        if(offset > file.length()) {
            offset = file.length();
        }
        return offset;
    }

I used corkami as a source to get to know some of the odds in calculating the overlay offset. I do not only want it to be robust, but also accurate. Did I miss something? What else do I have to put into consideration?

Note: There was a similar question here: How can one extract the appended data of a Portable Executable? But it doesn't cover a reliable algorithm so far. As I understand it, using a tool suffices in that question.

  • Do you care about the code-signature, if any? That's pointed to in one of the directories, but it's actually appended like an overlay. – 0xC0000022L May 13 '14 at 12:06
  • @0xC0000022L Good question. If it is not in a section, I would define it as overlay. – Karsten Hahn May 13 '14 at 14:04
7

You appear to be missing the corner cases such as mis-aligned pointers (should round down) and sizes (should round up).

However, even the rounding has corner cases - the physical pointer should be rounded down to a multiple of 512, regardless of the value in the header, but the read size is rounded up by using a combination of the file alignment and 4kb. The virtual size is always rounded up to a multiple of 4kb, regardless of the value in the header.

Digital signatures must be overlays. This is a check that is enforced now by Windows, for security reasons. If not an overlay, the file won't load.

You need something like this (and filealign comes from the PE header):

        long pointerToRaw = section.get(POINTER_TO_RAW_DATA);
        long alignedpointerToRaw = pointerToRaw & ~0x1ff;
        long sizeOfRaw = section.get(SIZE_OF_RAW_DATA);
        long readsize = ((pointerToRaw + sizeOfRaw) + filealign - 1) & ~(filealign - 1)) - alignedpointerToRaw;
        readsize = min(readsize, (sizeOfRaw + 0xfff) & ~0xfff);
        long virtsize = section.get(VIRTUAL_SIZE);

        if (virtsize)
        {
            readsize = min(readsize, (virtsize + 0xfff) & ~0xfff);
        }

Then you have "alignedpointerToRaw" as the starting position, and "readsize" as the number of bytes in the section. Sum these to find the end of the section. You need to perform this calculation for all sections (because physical data might not be sequential in the file). The largest sum is the end of the image. Anything beyond that is overlay.

  • About the alignedPointerToRaw: The specification says it should be a multiple of fileAlignment. The default value of fileAlignment is 512, but let's say it is not set to the default. Is it still correct that the section starts at pointerToRaw & ~0x1ff? – Karsten Hahn May 14 '14 at 7:00
  • Another question about fileAlignment: Should I correct the fileAlignment to this: "The value should be a power of 2 between 512 and 64 K, inclusive." (pecoff spec)? Thanks a lot for your help. – Karsten Hahn May 14 '14 at 7:23
  • 1
    aligning PointerToRaw is 512, regardless of the value in the header. Really. For possible values of fileAlignment, it's a power of 2, up to 64kb, but including 1 (i.e. 2^^0). – peter ferrie May 14 '14 at 15:40
  • Should I round up/down the file alignment to a power of 2 if it isn't? – Karsten Hahn May 15 '14 at 7:35
  • 1
    the file simply won't load if it's not a power of 2. You don't need to worry about this case. – peter ferrie May 15 '14 at 15:38
3

For the sake of completeness: This is is the code I made based on Peter Ferrie's suggestions

/**
 * Calculates the beginning of the overlay
 * 
 * @return the file offset to the beginning of overlay
 */
public long getOverlayOffset() {
    SectionTable table = data.getSectionTable();
    OptionalHeader opt = data.getOptionalHeader();
    offset = 0L;
    List<SectionHeader> headers = table.getSectionHeaders();
    if(headers.size() == 0) {
        offset = file.length(); //offset for sectionless PE's
    }
    for (SectionTableEntry section : table.getSectionEntries()) {
        long alignedPointerToRaw = section.get(POINTER_TO_RAW_DATA)
                & ~0x1ff;
        long endPoint = getReadSize(section) + alignedPointerToRaw;
        if (offset < endPoint) {
            offset = endPoint;
        }
    }
    if (offset > file.length()) {
        offset = file.length();
    }
    return offset;
}

/**
 * Determines the the number of bytes that is read for the section.
 * 
 * @param section
 * @return section bytes that are read
 */
private long getReadSize(SectionTableEntry section) {
    long pointerToRaw = section.get(POINTER_TO_RAW_DATA);
    long virtSize = section.get(VIRTUAL_SIZE);
    long sizeOfRaw = section.get(SIZE_OF_RAW_DATA);
    long fileAlign = data.getOptionalHeader().get(
            WindowsEntryKey.FILE_ALIGNMENT);
    long alignedPointerToRaw = section.get(POINTER_TO_RAW_DATA) & ~0x1ff;
    long readSize = alignedUp(pointerToRaw + sizeOfRaw, fileAlign)
            - alignedPointerToRaw;
    readSize = Math.min(readSize,
            alignedUp(section.get(SIZE_OF_RAW_DATA), 0x1000));
    if (virtSize != 0) {
        readSize = Math.min(readSize,
                alignedUp(section.get(VIRTUAL_SIZE), 0x1000));
    }
    return readSize;
}

/**
 * Returns the value rounded up to a multiple of alignTo.
 * 
 * @param value
 * @param alignTo
 * @return value rounded up to a multiple of alignTo
 */
private long alignedUp(long value, long alignTo) {
    if (value % alignTo != 0) {
        value = (value + alignTo - 1) & ~(alignTo - 1);
    }
    return value;
}
  • Is the constant 0x1000 the value for SectionAlignment? According to MSDN, this is the default -- but since a value is provided, you might want to use that anyway. – usr2564301 May 14 '14 at 14:31
  • 3
    @Jongware, it's always 0x1000. This and the 512 are (currently) hard-coded in Windows, despite what the documentation says (note that this is all undocumented, so theoretically might change one day). Section alignment from the header is for memory allocation purposes, rather than for file content. – peter ferrie May 14 '14 at 15:42
  • @Veitch, that looks proper. – peter ferrie May 14 '14 at 15:52
1

Have you considered using Microsoft's code to do it? Reverse the loader just enough to find the pointer that you're looking for once the image is loaded, stop it from executing any actual code beyond parsing the PE. The offset to your desired pointer in ntdll or wherever will be version-specific but you'll get a super-accurate parser.

  • 1
    that parsing is done in kernel mode, not user mode, and you'd still lack the actual algorithm. Blindly executing code which can change (even if the format does not) is not recommended. – peter ferrie May 23 '14 at 15:45
  • I was asking for an algorithm, because I write a PE-Library. So this is not an option. – Karsten Hahn May 23 '14 at 20:16
  • Well a reference implementation obviously wont help then. – offbyseveral May 23 '14 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.