I am analyzing a piece of malware that has a sequence of instructions with

prefix repne

0047094C  |.  8B0D 34935E00       |MOV ECX,DWORD PTR DS:[5E9334]
00470952  |.  F2:                 |PREFIX REPNE:
00470953  |.  0F2AC1              |CVTPI2PS XMM0,MM1
00470956  |.  F2:                 |PREFIX REPNE:
00470957  |.  0F100D 88905E00     |MOVUPS XMM1,DQWORD PTR DS:[5E9088]
0047095E  |.  F2:                 |PREFIX REPNE:
0047095F  |.  0F5CC8              |SUBPS XMM1,XMM0
00470962  |.  F2:                 |PREFIX REPNE:
00470963  |.  0F2CD1              |CVTTPS2PI MM2,XMM1
00470966  |.  8915 5C925E00       |MOV DWORD PTR DS:[5E925C],EDX

what's strange however is when Im debugging, the instructions immediately following the prefix repne are skipped (when single stepped). My instincts tell me it is just junk code but I want to make sure that something else isn't happening as I have to rewrite this particular function (there's plenty more sse instructions that are used)

If anyone can shed some light, I'd appreciate it. Thank you

  • 2
    repne is mnemonic for "repeat while not equal" - so if the zero flag is set when execution arrives at that address, the instruction won't get executed. If the zero flag is clear, it may get executed up to n times with n being the value of ecx; if the instruction may change the zero flag, there may be less repetitions. So, if you can be sure ZF is set before your instructions execute, this is junk code, but the instructions not getting executed in a single debugger trace doesn't mean this is always the case. – Guntram Blohm Jul 29 '15 at 13:47

The prefixes are not junk - they are an override for the original instruction, to produce a new instruction. Your disassembler is apparently not aware of SSE2 instructions, and separates them instead. The disassembly should read like this: 0047094C |. 8B0D 34935E00 |MOV ECX,DWORD PTR DS:[5E9334] 00470952 |. F20F2AC1 |CVTSI2SD XMM0,ECX 00470956 |. F20F100D 88905E00 |MOVSD XMM1,DQWORD PTR DS:[5E9088] 0047095E |. F20F5CC8 |SUBSD XMM1,XMM0 00470962 |. F20F2CD1 |CVTTSD2SI EDX,XMM1 00470966 |. 8915 5C925E00 |MOV DWORD PTR DS:[5E925C],EDX

| improve this answer | |
  • you're absolutely right! I have been so busy looking too deeply that I hadn't even noticed that IDA Pro correctly disassembles the instructions. Olly and other debuggers seem to fall into the disassemblers which are not SSE2 aware and separating the prefix. Thank you! – gandolf Jul 29 '15 at 16:35
  • fwiw odbg201 and windbg both disassembles the stream correctly – blabb Jul 29 '15 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.