This chapter is from the book
This chapter is from the book
This chapter is from the book
Xen Boot Options
The Xen hypervisor and Domain0 actually gain their privileged position in the system during boot. At system boot time, a bootloader such as GRUB—the GNU GRand Unified Bootloader, is used to choose the software that will control the machine. Because it is the first software "introduced" to the hardware, the Xen hypervisor now has the power to grant control to or withhold control from any other software.
The Xen hypervisor is inserted in the GRUB menu and boots first. The hypervisor then starts Domain0. If other guest domains have been previously created, Domain0 can automatically start them by consulting configuration files that specify the proper settings. These configuration files are discussed in more detail in Chapter 6, "Managing Unprivileged Domains."
Listing 3.2 shows some excerpts from the GRUB configuration file on the LiveCD used in Chapter 2. On the Xen LiveCD, it is /boot/grub/menu.lst, but on some systems it may be in other locations such as /boot/grub/grub.conf or /etc/grub.conf. For the Debian, CentOS, and OpenSUSE Domain0 sections, notice that the same kernel and modules are loaded (/boot/xen-3.0.3.0.gz, /boot/vmlinuz-2.6.16.29-xen, and /boot/initrd-2.6.16-xen.img). This is the equivalent of booting the Xen hypervisor. However, the image passed in the first module line differs. For the Debian Domain0, it is rootfs.img; for the CentOS Domain0, it is centos.img; and for the OpenSUSE Domain0, it is opensuse.img. Also notice that in the last section for the Native Non-Xen kernel, the Xen hypervisor is not booted. Instead, the non-Xen kernel and modules, /boot/vmlinuz-2.6.16-2-686-smp and /boot/initrd-2.6.16-2-686-smp.img, are specified. Of course, in all cases, the kernel and modules are coming from the CD as the root rather than the hard drive.
Listing 3.2. Excerpt from the GRUB Configuration File on the Xen LiveCD
terminal console
timeout 10
default 0
title Debian-based Dom0 (from testing)
root (cd)
kernel /boot/xen-3.0.3.0.gz watchdog
module /boot/vmlinuz-2.6.16.29-xen ro selinux=0
ramdisk_size=32758 image=rootfs.img boot=cow quiet
module /boot/initrd-2.6.16-xen.img
title CentOS-4.1-based Dom0
root (cd)
kernel /boot/xen-3.0.3.0.gz watchdog
module /boot/vmlinuz-2.6.16.29-xen ro selinux=0
ramdisk_size=32758 image=centos.img boot=cow
hotplug=/sbin/hotplug quiet
module /boot/initrd-2.6.16-xen.img
title OpenSUSE-10.0-based Dom0
root (cd)
kernel /boot/xen-3.0.3.0.gz watchdog
module /boot/vmlinuz-2.6.16.29-xen ro selinux=0
ramdisk_size=32758 image=opensuse.img boot=cow mkdevfd
quiet
module /boot/initrd-2.6.16-xen.img
title Debian on Native Kernel 2.6.16 (from testing)
(text mode only)
root (cd)
kernel /boot/vmlinuz-2.6.16-2-686-smp ro 2 selinux=0
ramdisk_size=32758 image=rootfs.img boot=cow
initrd /boot/initrd-2.6.16-2-686-smp.img
Table 3.2 describes a number of additional options that can be used to configure Xen's behavior at boot. They are added directly to the Xen command line in the GRUB configuration file. One of the most important options is how much memory to allocate to Domain0. Domain0 specifies to the hypervisor how much memory is allocated to all other guest domains, but the Xen hypervisor can be told the amount of memory to allocate to Domain0 directly. Because Domain0 should be simple (running the Xen daemon and not much else), it shouldn't need as much memory as a normal guest domain would. Chapter 4, "Hardware Requirements and Installation of Xen Domain0," explains the installation and configuration of Domain0 in detail.
Table 3.2. Xen Boot Options
|
Option |
Category |
Description |
|
dom0_mem=X |
Standard Configuration |
Amount of memory to be allocated to Domain0. |
|
Example: dom0_mem=64M |
Units can be specified as B for bytes, K for kilobytes, M for megabytes, or G for gigabytes. If no units are specified, kilobytes is assumed. |
|
|
lapic |
Advanced Configuration |
Forces use of the local APIC even if disabled by the uniprocessor BIOS. |
|
nolapic |
Advanced Configuration |
Do not use the local APIC in a uniprocessor system even if enabled by the BIOS. |
|
apic=default |
Advanced Configuration |
Specifies the APIC platform. This is usually determined automatically. |
apic=bigsmp apic=es7000 apic=summit boot option" apic=es7000 apic=summit" apic=es7000 apic=summit |
||
console=vga console=com1 console=com2H console=com2L |
Advanced Configuration |
Specify where Xen console I/O should be sent. More than one destination may be chosen. For vga, use regular console for output and keyboard for input. |
|
Default is console=com1,vga |
For com1, use serial port com1. com2H and com2L allow a single port to be used for both console and debugging. For com2H, characters must have most significant bit (MSB) set. For com2L, characters must have MSB cleared. |
|
conswitch=Cx conswitch=C |
Advanced Configuration |
Used to specify a different character sequence to indicate that serial-console input should be switched between Xen and Domain0. C specifies the "switch character." To actually switch, press Ctrl+C three times for the specified C. |
|
Example: conswitch=a |
The default switch character is a. |
|
|
conswitch=ax |
This also controls whether input is automatically switched to Domain0 when Domain0 boots. If the switch character is followed by an x, autoswitching is disabled. Any other value or omitting the x entirely enables autoswitching. |
|
|
nosmp |
Advanced Configuration |
Disable SMP support. |
sched=credit sched=sedf |
Advanced Configuration |
Choose a CPU scheduler for Xen to use. Because Xen offers an API for new schedulers to be added, more scheduling choices may appear in the future. The credit scheduler is currently recommended. |
|
badpage=X,Y,... |
Advanced Configuration |
You can specify a list of physical memory pages not to use because they contain bad bytes. Of course, you could also replace the bad memory! |
|
Example: badpage= 0x111, 0x222,0x333 |
||
|
noirqbalance |
Advanced Configuration |
Disable software IRQ balancing and affinity. Can be used for systems that have hardware for IRQ-routing issues—why do the job twice in hardware and software? |
com1=<BAUD>,<DPS>,<IOBASE>, <IRQ> com2=<BAUD>,<DPS>,<IOBASE>, <IRQ> |
Advanced Configuration/Debugging |
Xen output can be sent to the serial Typically ports, com1 and com2. this is done for debugging purposes. |
|
Example: com1=auto, 8n1, 0x408, 5 com2=9600, 8n1 |
BAUD = the baud rate. DPS = Data, parity, and stop bits. IOBASE = I/0 port base. IRQ = Interrupt Request Line. |
|
|
If some configuration options are standard, the configuration string can be abbreviated. |
||
|
mem=X |
Debugging/Testing |
Limit the amount of physical memory used in the system. Other than debugging/testing, you generally want to use all the memory you have in your system! Units can be specified as B for bytes, K for kilobytes, M for megabytes, or G for gigabytes. If no units are specified, kilobytes is assumed. |
|
tbuf_size=X |
Debugging |
Request X pages of per-cpu trace buffer space. Traces will be generated in debug builds only. |
|
noreboot |
Debugging |
Requests that the machine not be automatically rebooted in the case of fatal errors. |
|
sync_console |
Debugging |
Requests that all console output be sent immediately rather than buffered. Useful for reliably catching the last output generated by a fatal error. |
|
watchdog |
Debugging |
Enable the Non-Maskable Interrupt (NMI) watchdog timer, which can be helpful in reporting certain failures. |
|
nmi=fatal |
Debugging |
Specify what to do with an Non-Maskable Interrupt (NMI), parity, or I/O error. |
nmi=dom0 nmi=ignore |
For fatal, prints a diagnostic message and hangs. For Dom0, informs Domain0 of the error. For ignore, ignores the error. |
|
apic_verbosity=debug apic_verbosity=verbose |
Debugging |
Requests that information about the local APIC and IOAPIC be printed. |
Many of the hypervisor's configuration options control logging, such as the kind of output produced. You can request that the hypervisor produce a substantial amount of debugging information in the form of log messages or even traces of its activity. This output can be sent directly to the screen and/or to one of the serial ports in the system. This is most useful in diagnosing problems at boot. See the Debugging category of options in Table 3.2.
Another configuration option of note specifies what scheduling policy the hypervisor uses. Xen makes changing the scheduling policy easy. A number of different scheduling policies are currently available, including CREDIT and SEDF. SEDF provides weighted CPU sharing and includes real-time algorithms to ensure time guarantees. The CREDIT scheduler is a proportional fair share scheduler.
The CREDIT scheduler was implemented last but is generally recommended. It load balances virtual CPUs of the guest domains across all available physical CPUs. It is a work-conserving scheduler, meaning that physical devices will be idle only when there is no work to do. In contrast, a non-work-conserving scheduler may allow physical devices to be idle even when there is work to do if the idleness helps make the system more predictable.
The CREDIT scheduler allows each domain to be assigned both a weight and a cap. Domains with higher weights get more physical CPU time. Caps are used to limit the amount of CPU a domain can consume.
Other advanced configuration options include whether Symmetric Multiprocessing (nosmp) is supported, whether software IRQ balancing should be done (noirqbalance), and how the Advanced Programmable Interrupt Controller should be handled (apic). There are also some options to override the APIC settings in your BIOS settings. As you might imagine, not all of these options are applicable to every machine. Table 3.2 summarizes the available configuration options.
Table 3.3 contains options that, when specified on the Xen command line in the grub.conf file, will also be propagated to Domain0's command line when it boots. These options have the same syntax as a regular Linux kernel command line. Domain0's kernel is also modified to support one additional nonstandard boot option (xencons or Xen consoles), which is also described in Table 3.3.
Table 3.3. Xen-Related Linux Kernel Command Line Options
|
Linux Option |
Description |
|
acpi=off,force,strict,ht,noirq,... |
Specify how the Xen hypervisor and Domain0 should parse the BIOC ACPI tables. |
|
acpi_skip_timer_override |
Specify that Xen and Domain0 should ignore the timer-interrupt override instructions specified in the BIOS ACPI tables. |
|
noapic |
Specify that Xen and Domain0 should use the legacy PIC and ignore any IOAPICs present in the system. |
xencons=off xencons=tty xencons=ttyS |
Enable or disable the Xen virtual console. If enabled, specify the device node to which it is attached. For tty, attach to /dev/tty1; tty0 at boot time. For ttyS, attach to /dev/ttyS0. The default is ttyS for Domain0 and tty for DomUs. |
The GRUB configuration file from the Xen LiveCD did not use many of options described in Tables 3.2 and 3.3 (only the watchdog option). In Listing 3.3, we show how the Debian-based Domain0 section of the GRUB configuration file shown in Listing 3.2 could be modified to use the dom0_mem and xencons options.
Listing 3.3. GRUB Configuration File Illustrating Use of Additional Boot Options
title Debian-based Dom0 (from testing)
root (cd)
kernel /boot/xen-3.0.3.0.gz dom0_mem=65536 watchdog
module /boot/vmlinuz-2.6.16.29-xen ro selinux=0
ramdisk_size=32758 xencons=off image=rootfs.img
boot=cow quiet
module /boot/initrd-2.6.16-xen.img