Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Calling All Kernels
Every protected mode operating system needs some way of allowing userspace programs to perform privileged operations. This is typically done via system calls, where the kernel performs some permission checks and then performs a privileged operation on behalf of the process.
As with so many other things, x86 provides a few ways of doing system calls. On Windows 3.x, running DOS applications, programs simply executed an illegal instruction. It seems that the illegal instruction trap was the fastest way of jumping from virtual 8086 mode to protected mode.
Most operating systems now use interrupt 80h, which jumps to the interrupt handler in ring 0. This then reads the system call number from a register (usually EAX) and jump to the correct handler. Starting with the Pentium II, a faster mechanism was introduced, where the SYSENTER and SYSEXIT instructions could be used to quickly jump between ring 0 and ring 3.
Three ways of doing something seems a bit tame by x86 standards, but fortunately there is a fourth. The "standard" way of performing ring transitions is via a call gate. This is a function call address with some meta-data associated with it in the Global Descriptor Table (GDT) defining the ring transition to occur when the address is used as a destination for a CALL FAR instruction. This is the only mechanism that can be used to easily enter rings 1 and 2. It also provides some fine-grained control; rather than needing system call numbers in a register, system calls can simply be exposed as functions with known addresses and CALL'd directly. Call gates, being part of the segmented architecture, were removed with x86-64.