Reverse Engineering Stack Exchange is a question and answer site for researchers and developers who explore the principles of a system through analysis of its structure, function, and operation. It only takes a minute to sign up.
Sign up to join this community
I try to understand the process of memory segmentation for the i386 and amd64 architectures on Linux. It seems that this is heavily related to the segment registers %fs, %gs, %cs, %ss, %ds, %es.
Can somebody explain how these registers are used, both, in user and kernel-land programs ?
Kernel perspective:
I will try to answer from the kernel perspective, covering various OS's.
Memory segmentation is the old way of accessing memory regions. All major operating systems including OSX, Linux, (from version 0.1) and Windows (from NT) are now using paging which is a better way (IMHO) of accessing memory.
Intel, has always introduced backward compatibility in its processors (except IA-64, and we saw how it failed...) So, in its initial state (after reset) the processor starts in a mode called real mode, in this mode, segmentation is enabled by default to support legacy software. During the boot process of the operating system, the processor is changed into protected mode, and then in enabled paging.
Before paging, the segment registers were used like this
In real mode each logical address points directly into physical memory location, every logical address consists of two 16 bit parts: The segment part of the logical address contains the base address of a segment with a granularity of 16 bytes, i.e. a segments may start at physical address 0, 16, 32, ..., 220-16. The offset part of the logical address contains an offset inside the segment, i.e. the physical address can be calculated as
physical_address := segment_part × 16 + offset(if the address line A20 is enabled), respectively(segment_part × 16 + offset`) mod 220(if A20 is off) Every segment has a size of 216 bytes. [Wikipedia]
Let's see some examples (286-386 era) :
The 286 architecture introduced 4 segments: CS (code segment) DS (data segment) SS (stack segment) ES (extra segment) the 386 architecture introduced two new general segment registers FS, GS.
typical assembly opcode (in Intel syntax) would look like:
mov dx, 850h
mov es, dx ; Move 850h to es segment register
mov es:cx, 15h ; Move 15 to es:cx
Using paging (protected mode) the segment registers weren't used anymore for addressing memory locations.
In protected mode the
segment_partis replaced by a 16 bit selector, the 13 upper bits (bit 3 to bit 15) of the selector contains the index of an entry inside a descriptor table. The next bit (bit 2) specifies if the operation is used with the GDT or the LDT. The lowest two bits (bit 1 and bit 0) of the selector are combined to define the privilege of the request; where a value of 0 has the highest priority and value of 3 is the lowest. [wikipedia]
The segments however still used to enforce hardware security in the GDT
The Global Descriptor Table or GDT is a data structure used by Intel x86-family processors starting with the 80286 in order to define the characteristics of the various memory areas used during program execution, including the base address, the size and access privileges like executability and writability. These memory areas are called segments in Intel terminology. [wikipedia]
So, in practice the segment registers in protected mode are used to store indexes to the GDT.
Several operating systems such as Windows and Linux, use some of the segments for internal usage. for instance Windows x64 uses the GS register to access the TLS (thread local storage) and in Linux it's for accessing cpu specific memory.
User perspective:
From the user perspective, in recent operating system that uses paging, the memory works in so called "flat mode". Every process access its own memory (4GB) in linear fashion, so basically the segment registers are not needed.
They are still registers, so they can of course be used for various other assembly operations.
All major operating system ... use paging. I don't know about linux, but as you later pointed out in your answer, NT still use segments, for protection (on x86 at least). Privilege levels, or rings, are linked to segments. Hence the subdivision of the address space in (at least) four segments. The address space still look like 4gb flat, but the lower 2 (or 3) GB are one segment with ring 3 access, the upper 2(or 1) is a segment with ring 0 access.
May 23, 2013 at 8:45
mov es, 850h; Move 850h to es segment registerThis instruction above does not exist. Because it is not possible to load a segment register with an immediate value. It is only possible to load a segment register with a content of an other non segment register, with the content of a ram location, or with popping a value from our stack mov es:cx, 15h ; Move 15 to es:cx This instruction above also does not exist. Because we can not use CX as an adress register, we can only use ECX as an adress register.
FS points to the exception handling chain, CS and DS are filled from the OS with code and data segment. SS is the battery/stack segment. From what I remember, GS and ES are free.
It shouldn't matter much if kernel or user mode (they are used by some instructions like XLAT, MOVS, and some others, so you have to use them in the same way), but just in case I'm talking about programming in user space.
I had not noticed before, but you're using the notation %fs, not FS, so probably you're meaning Linux, which is another story (also you could be more clear on protected/real mode). You can see also from other answers on stackexchange that linux apparently gives you, in FS and GS, 'thread local storage' and 'processor data area'. CS, DS, and SS should still be code/data/stack.
For the sake of the argument, I have no idea how on a Mac you use those registers.
For 64 bit it depends: if not in compatibility mode (where you can execute 64 and 32 bit code) then DS, ES, and SS are ignored, and instructions like POP SS give an error. There is no segmentation (the memory model is flat), there should be no real mode (but I think you only mean protected mode?), and if I'm not wrong there isn't hardware task switching.
There are further details on CS, FS, and GS (expecially the hidden part) in 64 bit mode, but since it's not used often maybe it's better to omit them.
You can check the manuals for the AMD family of processors especially in the case of 64 bit legacy mode:
http://developer.amd.com/resources/documentation-articles/developer-guides-manuals/
i wrote a windows specific answer to a question that was marked as duplicate and closed and the close flag referred to this thread so i post an answer here
os win7 sp1 32 bit machine
kernel dump using livekd from sysinternals
a 16 bit segment register contains
13 bits of selector
1 bit of table descriptor
2 bits of requester_privilege_level
Selector tl rpl
0000000000000----0---00
so cs and fs converted to binary will be
kd> r cs;r fs
cs=00000008 = 0b 00001 0 00
fs=00000030 = 0b 00110 0 00
2 bits rpl means 0,1,2,3 rings ( so 00 = 0 = ring zero)
gdt = 1 bit means 0,1 (0 is for GDT and 1 is for LDT)
global descriptor table and local descriptor table
the high 13 bits represent segment selector
so cs = 0x08 has a segment selector of 0b 001 = 0x1 ie gdtr@1
& fs = 0x30 has a segment selector 0f 0b 110 = 0x6 ie gdtr@6
the kernel cs,fs are different from user cs,fs as can be noticed from dg command from windbg
kd> dg @cs <<<<<<<--- kernel
P Si Gr Pr Lo
Sel Base Limit Type l ze an es ng Flags
---- -------- -------- ---------- - -- -- -- -- --------
0008 00000000 ffffffff Code RE Ac 0 Bg Pg P Nl 00000c9b
0:000> dg @cs <<<<<<<<----user
P Si Gr Pr Lo
Sel Base Limit Type l ze an es ng Flags
---- -------- -------- ---------- - -- -- -- -- --------
001B 00000000 ffffffff Code RE Ac 3 Bg Pg P Nl 00000cfb
kd> dg @fs <<<<<<<<------- kernel
P Si Gr Pr Lo
Sel Base Limit Type l ze an es ng Flags
---- -------- -------- ---------- - -- -- -- -- --------
0030 82f6dc00 00003748 Data RW Ac 0 Bg By P Nl 00000493
0:000> dg @fs
P Si Gr Pr Lo
Sel Base Limit Type l ze an es ng Flags
---- -------- -------- ---------- - -- -- -- -- --------
003B 7ffdf000 00000fff Data RW Ac 3 Bg By P Nl 000004f3
you can glean sufficient information about gdt from
osdevwiki_gdt
robert-collins_ddj_article
to do that manually im using livekd here
using windbg you can get the Descriptor and Task Gate Registers
kd> rM 100
gdtr=80b95000 gdtl=03ff idtr=80b95400 idtl=07ff tr=0028 ldtr=0000
each gdtr entry is 64 bits so you can have 7f gdtr entries as you can see gdtl is 3ff 0x80*0x08-1 = 0x400-1 = 0x3ff (index starts from 0 not 1)
so gdtr entry @1,@2 are @gdtr+(0x1*0x8) @gdtr+(0x2*0x08=0x10) and so on
kd> dq @gdtr+8 l1 gdtr@1 = gdtr+0n1*0x8 =0n8 = 0x8
80b95008 00cf9b00`0000ffff = gdtr+0n6*0x8 =0n48 = 0x30
kd> dq @gdtr+30 l1
80b95030 824093f6`dc003748
kd> dq @gdtr+38 l1
80b95038 7f40f3fd`e0000fff
lets bit game the last two gdtr entries manually
-------------------------------------------------------------------------------------------
gdtrentry [63: [55: [51: [47: [39: [15:
56] 52] 48] 40] 16] 0]
base gdrs L p d t Base Base Limit
Hi rb0y h r l y Mid Low
-------------------------------------------------------------------------------------------
bit position 66665555 5555 5544 4 44 44444 33333333 3322222222221111 1111110000000000
32109876 5432 1098 7 65 43210 98765432 1098765432109876 5432109876543210
-------------------------------------------------------------------------------------------
824093f6dc003748 10000010 0100 0000 1 00 10011 11110110 1101110000000000 0011011101001000
as hex 0x82 0100 0 1 0 0x13 0xF6 0xDC00 0x3748
--------------------------------------- ---------------------------------------------------
7f40f3fde0000fff 01111111 0100 0000 1 11 10011 11111101 1110000000000000 0000111111111111
as hex 0x7F 0100 0 1 3 0x13 0xFD 0xE000 0x0FFF
-------------------------------------------------------------------------------------------
