What's the effect of adding a REP(N(E)) prefix to a JMP instruction?

Question

I'm attempting to reverse engineer an executable packer, and I'm a little stumped on this x86 instruction:

F2 EB F5    repne jmp short near ptr unk_88801B

According to the Intel manual a repeat prefix is not supported on a 'jmp' instruction. Yet IDA, Ollydbg and Capstone decode this instruction as the above.

This instruction appears in a section of code that is overlapped to obfuscate it, so I'm pretty certain the unsupported prefix is simply there to support the overlap.

My question is how does the processor handle this instruction? Does it just ignore the prefix, throw an illegal instruction, or ignore the instruction entirely?

Answer 1

This is a hard questions to answer since I have to explain how intel MTX works and I don't know how it works 100% but here I am going to try :)

At the beginning I though that IDA was giving you wrong disassemble code (that's not true I'll explain that later). Since you give me extra info (opcodes) I assemble it with rasm2 and I got this:

$ rasm2 -a x86 -d "F2 EB F5"
bnd jmp 0xfffffff8

BND! that is not a illegal instruction, actually it is call an Intel MPX (Memory Protection Extensions) It was first announced in 2013 and introduced in 2015.

From Wikipedia
Intel MPX is a set of extensions to the x86 instruction set architecture. With compiler, runtime library and operating system support, Intel MPX brings increased security to software by checking pointer references whose normal compile-time intentions are maliciously exploited at runtime due to buffer overflows.

Intel MPX provides four new registers named bnd0-bnd3 that are use to set bounds to avoid memory attack. I recommend you to read this, this, and this.

Getting back to your question. I think is going to work as a normal jmp because the bound registers are not initialized. How I know that? Your opcode is EB and if you read the articles you will read this.

An application compiled to use Intel MPX will use the REPNE (F2H) prefix (denoted by BND) for all forms of near CALL, near RET, near JMP, short & near Jcc instructions (BND+CALL, BND+RET, BND+JMP, BND+Jcc). See Table 17-4 for specific opcodes. All far CALL, RET and JMP instructions plus short JMP (JMP rel 8, opcode EB) instructions will never cause bound registers to be initialized

Reading that I also understand why IDA was not giving you wrong code.