XCP-XAPI

XCP-XAPI can be installed on top of other distributions (see Project Kronos), it includes the Xen hypervisor, installed at low level as the virtualization tool, with the XAPI toolstack used internally to manage the hypervisor.

xe is the command line tool used to configure and operate the Xen Cloud Platform (XCP) platform, while xsconsole being a TUI to do basic tasks and configuration.

It is possible to use the XCP-ISO distribution, but it comes on top of a CentOS system, just like Citrix's XenServer. Read the Xen / XCP / XAPI Overview to have a detailed view about the differences that exist between those.

XenServer is Citrix' commercial distribution of XCP.

Various management tools are available to use on top of XAPI.

Read the XCP Beginners Guide for more info.

A summarized version of the whole install process is available here: https://7terminals.com/articles/step-by-step-guide-to-setting-up-xen-and-xenapi-xcp-on-ubuntu-12-04-and-managing-it-with-xencenter/

Start with the base Debian installation as described in this wiki.

XCP requires lspci, part of the pciutils package, if you do not have the lspci command available:

> apt-get install pciutils

Then install XCP's xapi and all its dependencies, including the Xen hypervisor.

> apt-get update
> apt-get install xcp-xapi
   select: bridge network

Later, apt-get update && apt-get upgrade will keep up with the latest builds.

This is optional, but highly recommended on a server configuration.

> dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
> update-grub

Please refer to the Configure Networking section.

This was not applied and seems to work correctly up to now… Although we encounter great problems to modify this value later on, as far as our experience goes.

You should always dedicate a fixed amount of RAM for Xen dom0. For more details, see Xen - Configure Dom0 Memory.

Here are the commands to issue: Edit /etc/default/grub:

> nano /etc/default/grub

ADD:
# Xen boot parameters for all Xen boots
GRUB_CMDLINE_XEN_DEFAULT="dom0_mem=1280M,max:1280M dom0_max_vcpus=2"

Remember to apply the change to the grub configuration by running

> update-grub

Next step is to configure the toolstack to make sure dom0 memory is never ballooned down while starting new guests:

Using the XL toolstack, you should change /etc/xen/xl.conf

As we are unsing XAPI, which re-uses parts of the xend toolstack, we'll modify /etc/xen/xend-config.sxp, changing the “dom0-min-mem” option (to “dom0-min-mem 512”) and changing the “enable-dom0-ballooning” option (to “enable-dom0-ballooning no”). These options will make sure xend never takes any memory away from dom0. We'll then need to reboot the system.

> nano /etc/xen/xend-config.sxp

CHANGE:
line 208: (dom0-min-mem 1280)
line 212: (enable-dom0-ballooning no)

At this point you should reboot so that these changes take effect.

> reboot

> cat /proc/xen/capabilities

Should display “control_d”

At this stage the xcp-xapi service is not running as we didn't mention Xen to use it as its toolstack. Before we do this we still need to tweak some little settings in the system.

XAPI won't start yet as trying to start the service will yield an error:

[FAIL] /var/run/xend.pid exists; xapi conflicts with xend ... failed!

You will need to disable xend from starting in order to get xcp-xapi to start.

On the latest install (Debian Wheezy 7.4) /etc/init.d/xend does not exist!

This is done by modifying the file at /etc/init.d/xend. Many of the setup processes in this script still need to run, but xend itself should not start. This will be resolved in a future release. The command to resolve this is:

> sed -i -e 's/xend_start$/#xend_start/' -e 's/xend_stop$/#xend_stop/' /etc/init.d/xend

> update-rc.d xendomains disable

This will probably yield two warnings (no further explanation on this at the moment)

insserv: warning: current start runlevel(s) (empty) of script 'xendomains' overrides LSB defaults (2 3 4 5).
insserv: warning: current stop runlevel(s) (0 1 2 3 4 5 6) of script 'xendomains' overrides LSB defaults (0 1 6).

In order for vncterm to start up with a Linux VM, it needs to load the qemu keymaps however they are located in a location other than where it looks, this can be resolved with the following commands:

> ln -s /usr/share/qemu-linaro/keymaps /usr/share/qemu/keymaps

You can check whether xapi is running using the following command:

> service xcp-xapi status

If you are following this install process here's what you'll get:

[FAIL] Xen toolstack is not set to xapi! Exiting. ... failed! 

Set a TOOLSTACK variable to get xcp-xapi to start, you can do that with the following:

> nano /etc/default/xen

ADD:
TOOLSTACK=xapi

Configure xcp to use bridge networking or openswitch

> nano /etc/xcp/network.conf

REPLACE: “openvswitch” with “bridge” (or vice-versa).

After making those changes a reboot will be needed

> reboot

It is often interesting to setup at least one VM to autostart at boot time.

Do do so you can follow this next procedure, that is also described on this post:

> xe pool-list

uuid ( RO)                : 4a187cc1-69ce-eaf3-2742-6aec0783159f
          name-label ( RW): 
    name-description ( RW): 
              master ( RO): 288efd1c-7afe-21ca-e374-cace5e2d7e20
          default-SR ( RW): 26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0

> xe pool-param-set uuid=4a187cc1-69ce-eaf3-2742-6aec0783159f other-config:auto_poweron=true

> xe vm-list 
uuid ( RO)           : 76868e4b-4d82-320e-dd46-41340e6a67f3
     name-label ( RW): Control domain on host: store
    power-state ( RO): running


uuid ( RO)           : b7e42681-56c1-24cc-db45-9577981000b1
     name-label ( RW): XOA 3.6 Basic
    power-state ( RO): running

> xe vm-param-set uuid=b7e42681-56c1-24cc-db45-9577981000b1 other-config:auto_poweron=true

The previous steps are documented to be enough when using Citrix's XenServer. As we're using XCP/XAPI, one supplementary step is required, having a script to start at boot time all vms with “auto_poweron” in their other-config. This can be achieved by editing the /etc/rc.local file, adding the following code:

> nano /etc/rc.local

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

[ -e /proc/xen ] || exit 0

XAPI_START_TIMEOUT_SECONDS=240

# wait for xapi to complete initialisation for a max of XAPI_START_TIMEOUT_SECONDS
/usr/lib/xcp/bin/xapi-wait-init-complete ${XAPI_START_TIMEOUT_SECONDS}

if [ $? -eq 0 ]; then

pool=$(xe pool-list params=uuid --minimal 2> /dev/null)

auto_poweron=$(xe pool-param-get uuid=${pool} param-name=other-config param-key=auto_poweron 2> /dev/null)
if [ $? -eq 0 ] && [ "${auto_poweron}" = "true" ]; then
logger "$0 auto_poweron is enabled on the pool-- this is an unsupported configuration."

# if xapi init completed then start vms (best effort, don't report errors)
xe vm-start other-config:auto_poweron=true power-state=halted --multiple >/dev/null 2>/dev/null || true
fi
fi

You can test that the code works using the following command, it is important to validate its function as line 19 may need to be adapted, i.e: YMMV for /usr/lib/xcp/bin/xapi-wait-init-complete

> /etc/rc.local

The following procedure is based on the one described in this post.

First identifying our SR of choice, then create the file based VHD (VDI).

> xe sr-list
...
uuid ( RO)                : 26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0
          name-label ( RW): X-Local-SR
    name-description ( RW): 
                host ( RO): store
                type ( RO): ext
        content-type ( RO): 
...
  
> xe vdi-create sr-uuid=26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0 name-label=IPFire type=user virtual-size=5GiB
72e00fc6-98bb-48fe-ab4d-b52d1ef721b5

We'll now have a new vhd file at:
/run/sr-mount/<sr-uuid>/<vdi-uuid>.vhd

Which, in our example, translates to:
/run/sr-mount/26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0/72e00fc6-98bb-48fe-ab4d-b52d1ef721b5.vhd

The newly created VBD will link our Dom0 to the VDI we just created, first let's recover our Dom0 UUID:

> xe vm-list | grep -B 1 -e Control

uuid ( RO)           : 76868e4b-4d82-320e-dd46-41340e6a67f3
     name-label ( RW): Control domain on host: store

Then let's create and link the VBD to our Dom:

> xe vbd-create vm-uuid=76868e4b-4d82-320e-dd46-41340e6a67f3 vdi-uuid=72e00fc6-98bb-48fe-ab4d-b52d1ef721b5 device=autodetect

7472f458-ba3f-7344-99a7-6660a39037a6

Finaly we'll plug the VBD, making the VDI effectively available from the Dom0:

> xe vbd-plug uuid=7472f458-ba3f-7344-99a7-6660a39037a6

At this stage, a new device will get listed under /dev/sm/backend/<sr-uuid>/<vdi-uuid> like:

Before vdb-plug instruction:

ls -l /dev/sm/backend/26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0/
total 0
brw------- 1 root root 253, 0 Mar 29 01:38 4ea98b0d-b3fc-4f62-86bd-19a80f7d356b

After vdb-plug instruction:

ls -l /dev/sm/backend/26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0/
total 0
brw------- 1 root root 253, 0 Mar 29 01:38 4ea98b0d-b3fc-4f62-86bd-19a80f7d356b
brw------- 1 root root 253, 1 Mar 29 01:59 72e00fc6-98bb-48fe-ab4d-b52d1ef721b5

The kpartx command creates device maps from partition tables. Each guest storage image has a partition table embedded in the file.

We'll first need to install the package if it is not already available.

> apt-get install kpartx

kpartx lets you inspect an img file, showing its contained partitions using the -l command:

> kpartx -l /opt/xen/X-Local-ISO/<image_file_name>.img
loop0p1 : 0 122880 /dev/loop0 8192
loop0p3 : 0 1536000 /dev/loop0 131072
loop deleted : /dev/loop0

THIS DOESN'T WORK!

> kpartx -av /run/sr-mount/26b9d87b-f344-1c8d-c5c5-a155d4e4e2e0/72e00fc6-98bb-48fe-ab4d-b52d1ef721b5.vhd

X11 & VNC viewer

> apt-get install xorg

> cd /etc/X11/
> Xorg --configure

> startx

> echo $DISPLAY

> apt-get install xvnc4viewer

> reboot

> [sudo] xe vm-list

uuid ( RO) : dbcf74d2-ee50-edd5-d44d-b81fc8ba1777
name-label ( RW): Control domain on host: ubuntu-xenserver-1
power-state ( RO): running