I am reversing a file which is not running properly in Vmware.The code from the AEP is as shown below :

POP EDI     ; value of edi is 0x7C816D4F kernel32.7C816D4F
PUSH EAX    ;value of eax is 0
INC EBP     ;value of ebp was 0x12FFF0
IN EAX,DX   ;value of DX is 0xEB94    
IN AL,0BF                      

What I think is that a privileged instruction(IN) is called from user mode which is not allowed and therefore execution fails. IN is used for anti VM code but it requires specific values (VMXh port value in EAX etc) but in my case it is not being used.

My question is ,is it some kind of anti debugging or is the file corrupt and will it run on a non VM machine(in my case XP).

And lastly,if a packer uses the method I mentioned above ie calling IN from usermode for Anti-reversing how come the sample runs on a real machine(since in this case also the privileged instruction will be called in user mode).


2 Answers 2


Not easy to answer since there's not much context provided, but a few possibilities:

  1. The unpacking stub detected the VM at some point before executing the code above, and due to the VM detection, the unpacking stub caused the above junk code to be executed.
  2. The unpacking stub set up an exception handler before executing the code above, and the exception handler expects and catches the privileged instruction exception.
  3. This code is never executed and you're looking at the wrong memory.
  • 1
    and the first three characters spell "_PE". I doubt that it's executed code. Port 0xBF is not defined by standard hardware. Commented Mar 1, 2015 at 4:50
  • Thanks for the answer Jason: 1>This code is at the AEP and there is no TLS code present so I doubt that VM detection is possible. 2>There is no SEH routine barring the default one i.e. there is only one entry in the SEH chain 3>This code must run because it these are the instructions present at the AEP.
    – rebel87
    Commented Mar 2, 2015 at 4:59
  • Thanks for the comment Peter,could you elaborate how did you come to this conclusion...
    – rebel87
    Commented Mar 2, 2015 at 5:07
  • 1
    @rebel87: aas is an extremely rare opcode to see in modern compiled code; so is, dec esp -- subtracting only 1 from the stack pointer. Seeing 'random' disassembly such as this is an immediate red flag.
    – Jongware
    Commented Mar 4, 2015 at 10:13
  • @Jongware Thanks for the answer ,I'll keep that in mind i.e. Rare opcode is suspectible...
    – rebel87
    Commented Mar 4, 2015 at 10:31

I think I got the answer from : https://stackoverflow.com/questions/89607/what-is-a-privileged-instruction

Summary of the answers in that post :

The cause is probably a corrupted stack or a messed up function pointer call ,this usually happens when using function pointers that point to invalid data. It can also happen if you have code that trashes your return stack or if you are using old compilers/libraries.

The guy who programmed the exe may be using a local array and it is near the top of the function declaration. His bounds checking may have gone insane and overwritten the return address and it points to some instruction that only kernel is allowed to execute.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.