The arguments being passed to the open syscall at runtime in a program I'm debugging seem to be as follows:
open("SOMESTRING", 0xa01, 0x1b6);
These were retrieved using lldb on OSX 12.1 Monterey by setting a breakpoint at the open syscall and then printing out the args like this:
(lldb) x/s $rdi
0x6000022b4150: "SOMESTRING"
(lldb) p/x $rsi
(unsigned long) $5 = 0x0000000000000a01
(lldb) p/x $rdx
(unsigned long) $6 = 0x00000000000001b6
Using 'man 2 open', I'm reading a description of the arguments and then I'm going to the appropriate header files to check the hex value of each flag to try to determine what the parameters mean. The problem is, nothing seems to be matching up.
From the open manpage:
The flags specified for the oflag argument must include exactly one of the following file access modes:
O_RDONLY open for reading only
O_WRONLY open for writing only
O_RDWR open for reading and writing
In addition any combination of the following values can be or'ed in oflag:
O_NONBLOCK do not block on open or for data to become available
O_APPEND append on each write
O_CREAT create file if it does not exist
O_TRUNC truncate size to 0
O_EXCL error if O_CREAT and the file exists
O_SHLOCK atomically obtain a shared lock
O_EXLOCK atomically obtain an exclusive lock
O_DIRECTORY restrict open to a directory
O_NOFOLLOW do not follow symlinks
O_SYMLINK allow open of symlinks
O_EVTONLY descriptor requested for event notifications only
O_CLOEXEC mark as close-on-exec
O_NOFOLLOW_ANY do not follow symlinks in the entire path.
Now when I head over to fcntl.h, and look at the value for these flags, none of them have the hex value 0xa00. I'm failing to see how OR'ing any of these flags together would ever reproduce a value of 0xa01. I'm having similar trouble with the mode arg. Could anyone help me understand what I'm doing wrong?