Reverse Engineering Stack Exchange is a question and answer site for researchers and developers who explore the principles of a system through analysis of its structure, function, and operation. It only takes a minute to sign up.
Sign up to join this community
I'm completely new in malware reverse engineering. I'm stuck in a confusion that,
I studied that in x32, mul instruction multiplies the source with eax register and store the result as 64 bit value across two registers i.e. EDX:EAX. EDX stores most significant bit of 32 bits of operation and EAX do the same. So, my question is what happens in x64? how the values are stored in RAX:RDX after multiplication. I also ping the Google but did'nt understand much. I hope you all will respond.
According to the documentation Page 1320
The result is stored in register AX, register pair DX:AX, or register pair EDX:EAX (depending on the operand size), with the high-order bits of the product contained in register AH, DX, or EDX, respectively. If the high-order bits of the product are 0, the CF and OF flags are cleared; otherwise, the flags are set. In 64-bit mode, the instruction’s default operation size is 32 bits. Use of the REX.R prefix permits access to addi- tional registers (R8-R15). Use of the REX.W prefix promotes operation to 64 bits. See the summary chart at the beginning of this section for encoding data and limits.
It can be seen with this sequence of instructions
mov rdx, 0x1234567890abcdef
mov rax, 0x100
mul rdx
After the last instruction
$rax : 0x34567890abcdef00
$rdx : 0x12
RDX has the upper 64 bits as expected.
