Is there a way to write my own syscall/int 0x80 without using them?
So, normally it goes like
setup registers
...
syscall or int 0x80
and I am interested in doing this without syscall/int 0x80
setup registers
call mysyscall
where can I find implementation of syscall or int 0x80? Or is it too low level to implement it in asm?
TL;DR: These are two instructions supported by the CPU, you cannot implement them in assembly, as they are (or aren't) part of the assembly language you're using.
Some more background:
An interrupt (the assembly int instruction is causing a software interrupt) is a special event for the CPU. The CPU immediately (This is slightly inaccurate as advanced performance optimizations and low-level features are ignored for the sake of simplicity) stops executing the instruction sequence it was executing, saves the context of the current execution (such as the EIP and other control registers, most not directly accessible to the user), and switches to a specifically designated code sequence that is in charge of handling that specific interrupt.
Examples of hardware interrupts are related to handling power, hard-disk network and other peripherals, as well as when a program fails accessing a memory region, fails with certain calculations (divide by zero), floating point errors, tries executing a privileged or an invalid instruction and many others.
Additionally, a program can intentionally trigger an interrupt by using the int instruction which receives a single operand - the interrupt id to trigger.
When an interrupt happens, the resumes execution at the address pointed by the interrupt index in the Interrupt Descriptor Table (aka Interrupt Vector Table), in the case of int 0x80, that's the address at offset 0x80, obviously.
As interrupts are an expensive operation for the CPU (it entails a lot of bookkeeping related to theads and context switches, among other things), AMD (and later Intel) introduced the sysenter instruction (called syscall by Intel). That instruction keeps bookkeeping to the minimum by only raising to ring0 and executing kernel code.
