mprotect syscall not working

Question

I want to modify the protection of the .text segment to RWX using a shellcode. In the shellcode, I can issue the syscall for mprotect however it's not working.

Per the sys_mprotect definition, it takes the parameters as shown below:

rdi -> start_address
rsi -> length
rdx -> protection

Note: I have corrected the parameters above. The issue is not related to the parameter sequence.

so, I'm issuing the syscall as shown below:

push 0xA
pop rax
push 0x7
pop rdx
mov rdi, start_address
mov rsi, 0x1000
syscall

At the time of syscall, I verified that the registers are set properly.

start_address points to the main() subroutine inside the .text section which I want to modify.

However, the syscall does not modify the protection to RWX after execution. I verified the protection by viewing segments in IDA Pro.

Per the syscall_mprotect definition, the other registers don't have to be configured specifically for this syscall. So, I am not sure why this syscall is not working?

Update: I modified the start_address so that it's page aligned.

start_address = start_address - (start_address % 4096)

Now, syscall_mprotect does not return a negative value. It returns 0. However, the .text segment still doesn't show the writable flag.

Answer 1

Your parameters for mprotect are not correct. Here's the correct order

start: rdi
len:   rsi
prot:  rdx

The shellcode that works is

push 0xA
pop rax
push 0x7
pop rdx
call x
x:pop rdi
and rdi, 0xfffffffffffff000
mov rsi, 0x1000
syscall

The call x x: pop rdi is used to get the current rip. Assuming the code is in .text this should work. Link for reference on syscalls x86 x64

Answer 2

IDA does not update memory map after each step since it can potentially take a long time. You can try to either use “run to”(F4) instead of stepping or force refresh by calling refresh_debugger_memory() IDC function.