Differences

This shows you the differences between two versions of the page.

--- hypervisor:base:xen [2014/03/24 23:39]
admin [Configure guest behaviour on host reboot]
+++ hypervisor:base:xen [2014/04/03 02:07] (current)
admin Links adapted because of a move operation
@@ Line 1: / Line 1: @@
 ====== Xen ======
+<WRAP center round important 60%>
+In case you want to install the full XCP-XAPI toolstack, don't go any further and read the [[xcp-xapi|XCP-XAPI]] section.
+</WRAP>
 [[https://wiki.debian.org/Xen]]
@@ Line 27: / Line 31: @@
 ===== Configure Networking =====
-In order to give network access to guest domains it is necessary to configure the domain 0 network appropriately. The most common configuration is to use a software bridge.
+Please refer to the [[:hypervisor:base:network|Network configuration]] section.
-It is recommended that you manage your own network bridge using the [[https://wiki.debian.org/BridgeNetworkConnections|Debian network bridge]]. The Xen wiki page [[http://wiki.xen.org/wiki/Host%20Configuration/Networking|Host Configuration/Networking]] also has some useful information. The Xen supplied network scripts are not always reliable and will be removed from a later version. They are disabled by default in Debian's packages.
+===== Reliable Setup =====
-If you have a router that assigns ip addresses through dhcp, the following is a working example of the **/etc/network/interfaces** file using bridge-utils software.
+To avoid random, unexplained performance issues and failures in a XEN virtualization environment the following configuration items should always be checked;  Correspondingly, before troubleshooting ANY XEN Host or VM issue verify the XEN Host configuration as follows:
-<code>
-> nano /etc/network/interfaces
-#The loopback network interface
+  * "Lock down" the amount of memory assigned to dom0.
-auto lo
+  * Set the amount of memory Dom0 is given at boot time
-iface lo inet loopback
+  * Disable ballooning
+  * Do not restrict dom0 to a single VCPU
+  * Only use Logical Volume Manager (LVM) on Dom0 or DomU, but not both
-iface eth0 inet manual
+More (for SUSE): [[http://www.novell.com/support/kb/doc.php?id=3559698]]
+==== Configure Dom0 Memory ====
-auto xenbr0
-iface xenbr0 inet dhcp
-   bridge_ports eth0
-#other possibly useful options in a virtualized environment
-  #bridge_stp off       # disable Spanning Tree Protocol
-  #bridge_waitport 0    # no delay before a port becomes available
-  #bridge_fd 0          # no forwarding delay
-</code>
-activate your changes like this:
-<code>
-> ifdown eth0
-> killall dhclient
-> ifup xenbr0
-> brctl show
-bridge name bridge id       STP enabled interfaces
-br-lan      8000.xxxxxxxxxxxx   no      eth0
-</code>
-You should see your new IP address on ifconfig br-lan, and you should still be able to ping out (e.g. ping 8.8.8.8 / and resolve: ping google.com).
-===== Configure Domain 0 Memory =====
 On a system which is dedicated to running Xen guests it is better to instead give dom0 some static amount of RAM and to disable ballooning, here 1024MB.
@@ Line 98: / Line 77: @@
 At this point you should **reboot** so that these changes take effect.
+=== Ensure that Xen is running ===
+<code>
+> cat /proc/xen/capabilities
+</code>
+Should display “control_d”
 ===== Configure guest behaviour on host reboot =====
@@ Line 104: / Line 88: @@
 XENDOMAINS_RESTORE=false
 XENDOMAINS_SAVE=""
+</code>
+===== Configure Boot Parameters =====
+You may also want to pass some boot parameters to Xen when starting up in normal or recovery mode. Add these variables to /etc/default/grub to achieve this:
+<code>
+# Xen boot parameters for all Xen boots
+GRUB_CMDLINE_XEN="something"
+# Xen boot parameters for non-recovery Xen boots (in addition to GRUB_CMDLINE_XEN)
+GRUB_CMDLINE_XEN_DEFAULT="something else"
+</code>
+Remember to apply the change to the grub configuration by running update-grub!
+More information on the available hypervisor command line options can be found in the [[http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html|upstream documentation]]. =====
+===== Configure PCI pass-through Parameters =====
+This information is incomplete for Squeeze and needs to be updated for Wheezy
+To enable PCI pass-through, you need to know the BDF (Bus, Device, Function) id of the device. This is obtained through the lspci command, with the output containing the BDF in the format: (BB:DD.F) at the start of the line. To hide a device from Dom0 you will need to pass these boot parameters to Xen when starting. For example if using a Dom0 with 512M of memory and two devices at 01:08.1 and 01:09.2, add these variables to **/etc/default/grub** to achieve this:
+<code>
+# Xen boot parameters for all Xen boots
+GRUB_CMDLINE_XEN="dom0_mem=512M pciback.hide=(01:08.1)(01:09.2)"
+# Xen boot parameters for non-recovery Xen boots (in addition to GRUB_CMDLINE_XEN)
+GRUB_CMDLINE_XEN_DEFAULT="something else"
+</code>
+For Squeeze use "pciback.hide" (kernels < 2.6.32.10), for Wheezy (I have not tested this yet) use "xen-pciback.hide"
+for Squeeze you need to pass all of the devices on the bus, eg to pass any device on the 01:DD.F bus, you have pass all of them: (01:08.1)(01:09.2)(01:09.3)etc.
+Remember to apply the change to the grub configuration by running
+<code>
+> update-grub
+</code>
+At least in Wheezy (not tested in Squeeze) the xen-pciback module needs to be configured through //modprobe.conf// and added to the initramfs additionally.
+Configure the xen-pciback module by adding a modprobe include file (e.g. **/etc/modprobe.d/xen-pciback.conf**) with the following content (given that the PCI device would be assigned to module e1000e normally):
+<code>
+install e1000e /sbin/modprobe xen-pciback; /sbin/modprobe --first-time --ignore-install e1000e
+options xen-pciback hide=(0000:03:00.0)
+</code>
+Add the xen-pciback module to initramfs by adding it to **/etc/initramfs/modules** and running
+<code>
+> update-initramfs -u afterwards.
+</code>
+Please note that pci-passthrough is broken when msi is enabled (default) in Linux kernels < 3.14. Use Linux kernel >= 3.14 in DomU/VM or set pci=nomsi for DomU/VM kernel as workaround. See the following thread for detailed information: [[http://thread.gmane.org/gmane.comp.emulators.xen.user/81944/focus=191437]]
+===== Enable Serial Console =====
+To get output from GRUB, the Xen hypervisor, the kernel and getty (login prompt) via both VGA and serial console to work, here's an example of the right settings on squeeze:
+Edit **/etc/default/grub** and add:
+<code>
+GRUB_SERIAL_COMMAND="serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1"
+GRUB_TERMINAL="console serial"
+GRUB_TIMEOUT=5
+GRUB_CMDLINE_XEN="com1=9600,8n1 console=com1,vga"
+GRUB_CMDLINE_LINUX="console=tty0 console=hvc0"
+</code>
+Here's what I used to configure the serial console (for a Supermicro X8STi-F motherboard with IPMI and SOL):
+<code>
+GRUB_CMDLINE_XEN="loglvl=all guest_loglvl=all com1=115200,8n1,0x3e8,5 console=com1,vga"
+GRUB_CMDLINE_LINUX="console=hvc0 earlyprintk=xen"
+</code>
+In **/etc/inittab** you need at least these lines:
+<code>
+:2345:respawn:/sbin/getty 38400 hvc0
+:23:respawn:/sbin/getty 38400 tty1
+# NO getty on ttyS0!
+</code>
+This way, tty1 will show up at the VGA output, and the hvc0 will show up at the serial console.
+To keep both Xen and dom0 kernel output on the same tty, just omit the "vga"-related settings from the above setup.
+If you need to debug Xen and see a crash dump of the kernel, you can do it using IPMITool if your server has SOL:
+<code>
+ipmitool -I lanplus -H server-ip-address -U your-username sol activate | tee my-log-file.txt
 </code>