Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
Applies to openSUSE Leap 15.1

14 Configuring Virtual Machines with virsh

Abstract

You can use virsh to configure the VM Guest in addition to using the Virtual Machine Manager. virsh is a command line tool that manages virtual machines created by VM software, such as KVM, Xen, VMware, and LXC. With virsh, you can control the state of a VM, edit the configuration of a VM or even migrate a VM to another host. The following section shows the management of a VM with virsh. Examples include modifying memory allocation, input devices, CPU settings, and video configuration.

14.1 Editing the VM Configuration

The configuration of a VM is stored in an XML file in /etc/libvirtd/qemu/ and looks like this:

Example 14.1: Example XML Configuration file
    <domain type='kvm'>
      <name>sles15</name>
      <uuid>ab953e2f-9d16-4955-bb43-1178230ee625</uuid>
      <memory unit='KiB'>2097152</memory>
      <currentMemory unit='KiB'>2097152</currentMemory>
      <vcpu placement='static'>2</vcpu>
      <os>
        <type arch='x86_64' machine='pc-i440fx-2.11'>hvm</type>
      </os>
      <features>...</features>
      <cpu mode='custom' match='exact' check='partial'>
        <model fallback='allow'>Skylake-Client-IBRS</model>
      </cpu>
      <clock>...</clock>
      <on_poweroff>destroy</on_poweroff>
      <on_reboot>restart</on_reboot>
      <on_crash>destroy</on_crash>
      <pm>
        <suspend-to-mem enabled='no'/>
        <suspend-to-disk enabled='no'/>
      </pm>
      <devices>
        <emulator>/usr/bin/qemu-system-x86_64</emulator>
        <disk type='file' device='disk'>...</disk>
      </devices>
      ...
    </domain>

If you want to edit the configuration of the VM Guest, check if the guest is offline:

tux > sudo virsh list --inactive

If your VM Guest is in this list, you can safely edit its configuration with:

tux > sudo virsh edit NAME_OF_VM_GUEST

Before saving the changes, virsh validates your input against a RelaxNG schema.

14.2 Managing Guest Memory Allocation (Xen only)

libvirt now includes support for adjusting memory allocation per guest with virsh. Xen paravirtual devices connect to the xenbus controller, which is analogous to a physical device bus such as a PCI controller. Xen's max_grant_frames attribute sets how many frames, which are analogous to memory, are allocated to the xenbus controller for each guest.

The default is 32, and this can be increased as needed, up to the amount set for dom0, or decreased. How much is enough? This depends on the number of devices and workload demand, such as a saturated network interface or heavy I/O. Use xen-diag to see your current used and maximum max_grant_frames values for dom0 and your guests. The guests must be running:

tux > sudo virsh list 
 Id   Name             State
--------------------------------
 0    Domain-0         running
 3    sle15sp1         running
 
 tux > sudo xen-diag gnttab_query_size 0
domid=0: nr_frames=1, max_nr_frames=256

tux > sudo xen-diag gnttab_query_size 3
domid=3: nr_frames=3, max_nr_frames=32

The sle15sp1 guest is using only 3 frames out of 32. If you are seeing performance issues, and log entries that point to insufficient frames, increase the value with virsh. Look for the <controller type='xenbus' line in the guest's configuration file and add the maxGrantFrames control element:

tux > sudo virsh edit sle15sp1
 <controller type='xenbus' index='0' maxGrantFrames='40'/>

Save your changes and restart the guest. Now it should show your change:

tux > sudo xen-diag gnttab_query_size 3
domid=3: nr_frames=3, max_nr_frames=40

See the Controllers section of the libvirt Domain XML format manual at https://libvirt.org/formatdomain.html#elementsControllers for more information.

14.3 Changing the Machine Type

When installing with the virt-install tool, the machine type for a VM Guest is pc-i440fx by default. The machine type is stored in the VM Guest's configuration file in the type element:

<type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>

As an example, the following procedure shows how to change this value to the machine type q35. The value q35 is an Intel* chipset and includes PCIe, supports up to 12 USB ports, and has support for SATA and IOMMU.

Procedure 14.1: Changing Machine Type
  1. Check whether your VM Guest is inactive:

    tux > sudo virsh list --inactive
    Id    Name                           State
    ----------------------------------------------------
    -     sles15                         shut off
  2. Edit the configuration for this VM Guest:

    tux > sudo virsh edit sles15
  3. Replace the value of the machine attribute with pc-q35-2.0 :

    <type arch='x86_64' machine='pc-q35-2.0'>hvm</type>
  4. Restart the VM Guest:

    tux > sudo virsh start sles15
  5. Check if the machine type has changed. Log in to the VM Guest and run the following command:

    tux > sudo dmidecode | grep Product
    Product Name: Standard PC (Q35 + ICH9, 2009)
Tip
Tip: Machine Type Update Recommendations

Whenever the QEMU version on the host system is upgraded (for example, when upgrading the VM Host Server to a new service pack), upgrade the machine type of the VM Guests to the latest available version. To check, use the command qemu-system-x86_64 -M help on the VM Host Server.

The default machine type pc-i440fx, for example, is regularly updated. If your VM Guest still runs with a machine type of pc-i440fx-1.X, we strongly recommend an update to pc-i440fx-2.X. This allows taking advantage of the most recent updates and corrections in machine definitions, and ensures better future compatibility.

14.4 Configuring CPU Allocation

The number of allocated CPUs is stored in the VM Guest's XML configuration file in /etc/libvirt/qemu/ in the vcpu element:

<vcpu placement='static'>1</vcpu>

In this example, the VM Guest has only one allocated CPU. The following procedure shows how to change the number of allocated CPUs for the VM Guest:

  1. Check whether your VM Guest is inactive:

    tux > sudo virsh list --inactive
    Id    Name                           State
    ----------------------------------------------------
    -     sles15                         shut off
  2. Edit the configuration for an existing VM Guest:

    tux > sudo virsh edit sles15
  3. Change the number of allocated CPUs:

    <vcpu placement='static'>2</vcpu>
  4. Restart the VM Guest:

    tux > sudo virsh start sles15
  5. Check if the number of CPUs in the VM has changed.

    tux > sudo virsh vcpuinfo sled15
    VCPU:           0
    CPU:            N/A
    State:          N/A
    CPU time        N/A
    CPU Affinity:   yy
    
    VCPU:           1
    CPU:            N/A
    State:          N/A
    CPU time        N/A
    CPU Affinity:   yy

14.5 Changing Boot Options

The boot menu of the VM Guest can be found in the os element and usually looks like this:

<os>
  <type>hvm</type>
  <loader>readonly='yes' secure='no' type='rom'/>/usr/lib/xen/boot/hvmloader</loader>
  <nvram template='/usr/share/OVMF/OVMF_VARS.fd'/>/var/lib/libvirt/nvram/guest_VARS.fd</nvram>
  <boot dev='hd'/>
  <boot dev='cdrom'/>
  <bootmenu enable='yes' timeout='3000'/>
  <smbios mode='sysinfo'/>
  <bios useserial='yes' rebootTimeout='0'/>
  </os>

In this example, two devices are available, hd and cdrom . The configuration also reflects the actual boot order, so the cdrom comes before the hd .

14.5.1 Changing Boot Order

The VM Guest's boot order is represented through the order of devices in the XML configuration file. As the devices are interchangeable, it is possible to change the boot order of the VM Guest.

  1. Open the VM Guest's XML configuration.

    tux > sudo virsh edit sles15
  2. Change the sequence of the bootable devices.

    ...
    <boot dev='cdrom'/>
    <boot dev='hd'/>
    ...
  3. Check if the boot order was changed successfully by looking at the boot menu in the BIOS of the VM Guest.

14.5.2 Using Direct Kernel Boot

Direct Kernel Boot allows you to boot from a kernel and initrd stored on the host. Set the path to both files in the kernel and initrd elements:

<os>
    ...
  <kernel>/root/f8-i386-vmlinuz</kernel>
  <initrd>/root/f8-i386-initrd</initrd>
    ...
<os>

To enable Direct Kernel Boot:

  1. Open the VM Guest's XML configuration:

    tux > sudo virsh edit sles15
  2. Inside the os element, add a kernel element and the path to the kernel file on the host:

    ...
    <kernel>/root/f8-i386-vmlinuz</kernel>
    ...
  3. Add an initrd element and the path to the initrd file on the host:

    ...
    <initrd>/root/f8-i386-initrd</initrd>
    ...
  4. Start your VM to boot from the new kernel:

    tux > sudo virsh start sles15

14.6 Configuring Memory Allocation

The amount of memory allocated for the VM Guest can also be configured with virsh. It is stored in the memory element. Follow these steps:

  1. Open the VM Guest's XML configuration:

    tux > sudo virsh edit sles15
  2. Search for the memory element and set the amount of allocated RAM:

    ...
    <memory unit='KiB'>524288</memory>
    ...
  3. Check the amount of allocated RAM in your VM by running:

    tux > cat /proc/meminfo

14.7 Adding a PCI Device

To assign a PCI device to VM Guest with virsh, follow these steps:

  1. Identify the host PCI device to assign to the VM Guest. In the following example, we are assigning a DEC network card to the guest:

    tux > sudo lspci -nn
    [...]
    03:07.0 Ethernet controller [0200]: Digital Equipment Corporation DECchip \
    21140 [FasterNet] [1011:0009] (rev 22)
    [...]

    Write down the device ID (03:07.0 in this case).

  2. Gather detailed information about the device using virsh nodedev-dumpxml ID. To get the ID, replace the colon and the period in the device ID (03:07.0) with underscores. Prefix the result with pci_0000_: pci_0000_03_07_0.

    tux > sudo virsh nodedev-dumpxml pci_0000_03_07_0
    <device>
      <name>pci_0000_03_07_0</name>
      <path>/sys/devices/pci0000:00/0000:00:14.4/0000:03:07.0</path>
      <parent>pci_0000_00_14_4</parent>
      <driver>
        <name>tulip</name>
      </driver>
      <capability type='pci'>
        <domain>0</domain>
        <bus>3</bus>
        <slot>7</slot>
        <function>0</function>
        <product id='0x0009'>DECchip 21140 [FasterNet]</product>
        <vendor id='0x1011'>Digital Equipment Corporation</vendor>
        <numa node='0'/>
      </capability>
    </device>

    Write down the values for domain, bus, and function (see the previous XML code printed in bold).

  3. Detach the device from the host system prior to attaching it to the VM Guest:

    tux > sudo virsh nodedev-detach pci_0000_03_07_0
      Device pci_0000_03_07_0 detached
    Tip
    Tip: Multi-Function PCI Devices

    When using a multi-function PCI device that does not support FLR (function level reset) or PM (power management) reset, you need to detach all its functions from the VM Host Server. The whole device must be reset for security reasons. libvirt will refuse to assign the device if one of its functions is still in use by the VM Host Server or another VM Guest.

  4. Convert the domain, bus, slot, and function value from decimal to hexadecimal. In our example, domain = 0, bus = 3, slot = 7, and function = 0. Ensure that the values are inserted in the right order:

    tux > printf "<address domain='0x%x' bus='0x%x' slot='0x%x' function='0x%x'/>\n" 0 3 7 0

    This results in:

    <address domain='0x0' bus='0x3' slot='0x7' function='0x0'/>
  5. Run virsh edit on your domain, and add the following device entry in the <devices> section using the result from the previous step:

    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0' bus='0x03' slot='0x07' function='0x0'/>
      </source>
    </hostdev>
    Tip
    Tip: managed Compared to unmanaged

    libvirt recognizes two modes for handling PCI devices: they can be either managed or unmanaged. In the managed case, libvirt handles all details of unbinding the device from the existing driver if needed, resetting the device, binding it to vfio-pci before starting the domain, etc. When the domain is terminated or the device is removed from the domain, libvirt unbinds from vfio-pci and rebinds to the original driver in the case of a managed device. If the device is unmanaged, the user must ensure all of these management aspects of the device are done before assigning it to a domain, and after the device is no longer used by the domain.

    In the example above, the managed='yes' option means that the device is managed. To switch the device mode to unmanaged, set managed='no' in the listing above. If you do so, you need to take care of the related driver with the virsh nodedev-detach and virsh nodedev-reattach commands. Prior to starting the VM Guest you need to detach the device from the host by running virsh nodedev-detach pci_0000_03_07_0. In case the VM Guest is not running, you can make the device available for the host by running virsh nodedev-reattach pci_0000_03_07_0.

  6. Shut down the VM Guest and disable SELinux if it is running on the host.

    tux > sudo setsebool -P virt_use_sysfs 1
  7. Start your VM Guest to make the assigned PCI device available:

    tux > sudo virsh start sles15

14.8 Adding a USB Device

To assign a USB device to VM Guest using virsh, follow these steps:

  1. Identify the host USB device to assign to the VM Guest:

    tux > sudo lsusb
    [...]
    Bus 001 Device 003: ID 0557:2221 ATEN International Co., Ltd Winbond Hermon
    [...]

    Write down the vendor and product IDs. In our example, the vendor ID is 0557 and the product ID is 2221.

  2. Run virsh edit on your domain, and add the following device entry in the <devices> section using the values from the previous step:

    <hostdev mode='subsystem' type='usb'>
      <source startupPolicy='optional'>
       <vendor id='0557'/>
       <product id='2221'/>
      </source>
    </hostdev>
    Tip
    Tip: Vendor/Product or Device's Address

    Instead of defining the host device with <vendor/> and <product/> IDs, you can use the <address/> element as described for host PCI devices in Section 14.7, “Adding a PCI Device”.

  3. Shut down the VM Guest and disable SELinux if it is running on the host:

    tux > sudo setsebool -P virt_use_sysfs 1
  4. Start your VM Guest to make the assigned PCI device available:

    tux > sudo virsh start sles15

14.9 Adding SR-IOV Devices

Single Root I/O Virtualization (SR-IOV) capable PCIe devices can replicate their resources, so they appear to be multiple devices. Each of these pseudo-devices can be assigned to a VM Guest.

SR-IOV is an industry specification that was created by the Peripheral Component Interconnect Special Interest Group (PCI-SIG) consortium. It introduces physical functions (PF) and virtual functions (VF). PFs are full PCIe functions used to manage and configure the device. PFs also can move data. VFs lack the configuration and management part—they only can move data and a reduced set of configuration functions. As VFs do not have all PCIe functions, the host operating system or the Hypervisor must support SR-IOV to be able to access and initialize VFs. The theoretical maximum for VFs is 256 per device (consequently the maximum for a dual-port Ethernet card would be 512). In practice this maximum is much lower, since each VF consumes resources.

14.9.1 Requirements

The following requirements must be met to use SR-IOV:

  • An SR-IOV-capable network card (as of , only network cards support SR-IOV)

  • An AMD64/Intel 64 host supporting hardware virtualization (AMD-V or Intel VT-x)

  • A chipset that supports device assignment (AMD-Vi or Intel VT-d)

  • libvirt 0.9.10 or better

  • SR-IOV drivers must be loaded and configured on the host system

  • A host configuration that meets the requirements listed at Important: Requirements for VFIO and SR-IOV

  • A list of the PCI addresses of the VF(s) that will be assigned to VM Guests

Tip
Tip: Checking if a Device is SR-IOV-Capable

The information whether a device is SR-IOV-capable can be obtained from its PCI descriptor by running lspci. A device that supports SR-IOV reports a capability similar to the following:

Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
Note
Note: Adding an SR-IOV Device at VM Guest Creation

Before adding an SR-IOV device to a VM Guest when initially setting it up, the VM Host Server already needs to be configured as described in Section 14.9.2, “Loading and Configuring the SR-IOV Host Drivers”.

14.9.2 Loading and Configuring the SR-IOV Host Drivers

To access and initialize VFs, an SR-IOV-capable driver needs to be loaded on the host system.

  1. Before loading the driver, make sure the card is properly detected by running lspci. The following example shows the lspci output for the dual-port Intel 82576NS network card:

    tux > sudo /sbin/lspci | grep 82576
    01:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    01:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    04:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    04:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)

    In case the card is not detected, it is likely that the hardware virtualization support in the BIOS/EFI has not been enabled. To check if hardware virtualization support is enabled, look at the settings in the host's BIOS.

  2. Check whether the SR-IOV driver is already loaded by running lsmod. In the following example a check for the igb driver (for the Intel 82576NS network card) returns a result. That means the driver is already loaded. If the command returns nothing, the driver is not loaded.

    tux > sudo /sbin/lsmod | egrep "^igb "
    igb                   185649  0
  3. Skip the following step if the driver is already loaded. If the SR-IOV driver is not yet loaded, the non-SR-IOV driver needs to be removed first, before loading the new driver. Use rmmod to unload a driver. The following example unloads the non-SR-IOV driver for the Intel 82576NS network card:

    tux > sudo /sbin/rmmod igbvf
  4. Load the SR-IOV driver subsequently using the modprobe command—the VF parameter (max_vfs) is mandatory:

    tux > sudo /sbin/modprobe igb max_vfs=8

As an alternative, you can also load the driver via SYSFS:

  1. Find the PCI ID of the physical NIC by listing Ethernet devices:

    tux > sudo lspci | grep Eth
    06:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)
    06:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)
  2. To enable VFs, echo the number of desired VFs to load to the sriov_numvfs parameter:

    tux > sudo echo 1 > /sys/bus/pci/devices/0000:06:00.1/sriov_numvfs
  3. Verify that the VF NIC was loaded:

    tux > sudo lspci | grep Eth
    06:00.0 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)
    06:00.1 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)
    06:08.0 Ethernet controller: Emulex Corporation OneConnect NIC (Skyhawk) (rev 10)
  4. Obtain the maximum number of VFs available:

    tux > sudo lspci -vvv -s 06:00.1 | grep 'Initial VFs'
                           Initial VFs: 32, Total VFs: 32, Number of VFs: 0,
    Function Dependency Link: 01
  5. Create a /etc/systemd/system/before.service file which loads VF via SYSFS on boot:

    [Unit]
    Before=
    [Service]
    Type=oneshot
    RemainAfterExit=true
    ExecStart=/bin/bash -c "echo 1 > /sys/bus/pci/devices/0000:06:00.1/sriov_numvfs"
    # beware, executable is run directly, not through a shell, check the man pages
    # systemd.service and systemd.unit for full syntax
    [Install]
    # target in which to start the service
    WantedBy=multi-user.target
    #WantedBy=graphical.target
  6. Prior to starting the VM, it is required to create another service file (after-local.service) pointing to the /etc/init.d/after.local script that detaches the NIC. Otherwise the VM would fail to start:

    [Unit]
    Description=/etc/init.d/after.local Compatibility
    After=libvirtd.service
    Requires=libvirtd.service
    [Service]
    Type=oneshot
    ExecStart=/etc/init.d/after.local
    RemainAfterExit=true
    
    [Install]
    WantedBy=multi-user.target
  7. Copy it to /etc/systemd/system.

    #! /bin/sh
    # ...
    virsh nodedev-detach pci_0000_06_08_0

    Save it as /etc/init.d/after.local.

  8. Reboot the machine and check if the SR-IOV driver is loaded by re-running the lspci command from the first step of this procedure. If the SR-IOV driver was loaded successfully you should see additional lines for the VFs:

    01:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    01:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    01:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    01:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    01:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    [...]
    04:00.0 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    04:00.1 Ethernet controller: Intel Corporation 82576NS Gigabit Network Connection (rev 01)
    04:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    04:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    04:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
    [...]

14.9.3 Adding a VF Network Device to a VM Guest

When the SR-IOV hardware is properly set up on the VM Host Server, you can add VFs to VM Guests. To do so, you need to collect some data first.

Procedure 14.2: Adding a VF Network Device to an Existing VM Guest

The following procedure uses example data. Make sure to replace it by appropriate data from your setup.

  1. Use the virsh nodedev-list command to get the PCI address of the VF you want to assign and its corresponding PF. Numerical values from the lspci output shown in Section 14.9.2, “Loading and Configuring the SR-IOV Host Drivers” (for example 01:00.0 or 04:00.1) are transformed by adding the prefix "pci_0000_" and by replacing colons and dots with underscores. So a PCI ID listed as "04:00.0" by lspci is listed as "pci_0000_04_00_0" by virsh. The following example lists the PCI IDs for the second port of the Intel 82576NS network card:

    tux > sudo virsh nodedev-list | grep 0000_04_
    pci_0000_04_00_0
    pci_0000_04_00_1
    pci_0000_04_10_0
    pci_0000_04_10_1
    pci_0000_04_10_2
    pci_0000_04_10_3
    pci_0000_04_10_4
    pci_0000_04_10_5
    pci_0000_04_10_6
    pci_0000_04_10_7
    pci_0000_04_11_0
    pci_0000_04_11_1
    pci_0000_04_11_2
    pci_0000_04_11_3
    pci_0000_04_11_4
    pci_0000_04_11_5

    The first two entries represent the PFs, whereas the other entries represent the VFs.

  2. Run the following virsh nodedev-dumpxml command on the PCI ID of the VF you want to add:

    tux > sudo virsh nodedev-dumpxml pci_0000_04_10_0
    <device>
      <name>pci_0000_04_10_0</name>
      <parent>pci_0000_00_02_0</parent>
      <capability type='pci'>
        <domain>0</domain>
        <bus>4</bus>
        <slot>16</slot>
        <function>0</function>
        <product id='0x10ca'>82576 Virtual Function</product>
        <vendor id='0x8086'>Intel Corporation</vendor>
        <capability type='phys_function'>
          <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
        </capability>
      </capability>
    </device>

    The following data is needed for the next step:

    • <domain>0</domain>

    • <bus>4</bus>

    • <slot>16</slot>

    • <function>0</function>

  3. Create a temporary XML file (for example /tmp/vf-interface.xml containing the data necessary to add a VF network device to an existing VM Guest. The minimal content of the file needs to look like the following:

    <interface type='hostdev'>1
     <source>
      <address type='pci' domain='0' bus='11' slot='16' function='0'2/>2
     </source>
    </interface>

    1

    VFs do not get a fixed MAC address; it changes every time the host reboots. When adding network devices the traditional way with hostdev, it would require to reconfigure the VM Guest's network device after each reboot of the host, because of the MAC address change. To avoid this kind of problem, libvirt introduced the hostdev value, which sets up network-specific data before assigning the device.

    2

    Specify the data you acquired in the previous step here.

  4. In case a device is already attached to the host, it cannot be attached to a VM Guest. To make it available for guests, detach it from the host first:

    tux > sudo virsh nodedev-detach pci_0000_04_10_0
  5. Add the VF interface to an existing VM Guest:

    tux > sudo virsh attach-device GUEST /tmp/vf-interface.xml --OPTION

    GUEST needs to be replaced by the domain name, ID or UUID of the VM Guest. --OPTION can be one of the following:

    --persistent

    This option will always add the device to the domain's persistent XML. In addition, if the domain is running, it will be hotplugged.

    --config

    This option will only affect the persistent XML, even if the domain is running. The device will only show up in the VM Guest on next boot.

    --live

    This option will only affect a running domain. If the domain is inactive, the operation will fail. The device is not persisted in the XML and will not be available in the VM Guest on next boot.

    --current

    This option affects the current state of the domain. If the domain is inactive, the device is added to the persistent XML and will be available on next boot. If the domain is active, the device is hotplugged but not added to the persistent XML.

  6. To detach a VF interface, use the virsh detach-device command, which also takes the options listed above.

14.9.4 Dynamic Allocation of VFs from a Pool

If you define the PCI address of a VF into a VM Guest's configuration statically as described in Section 14.9.3, “Adding a VF Network Device to a VM Guest”, it is hard to migrate such guest to another host. The host must have identical hardware in the same location on the PCI bus, or the VM Guest configuration must be modified prior to each start.

Another approach is to create a libvirt network with a device pool that contains all the VFs of an SR-IOV device. The VM Guest then references this network, and each time it is started, a single VF is dynamically allocated to it. When the VM Guest is stopped, the VF is returned to the pool, available for another guest.

14.9.4.1 Defining Network with Pool of VFs on VM Host Server

The following example of network definition creates a pool of all VFs for the SR-IOV device with its physical function (PF) at the network interface eth0 on the host:

<network>
  <name>passthrough</name>
    <forward mode='hostdev' managed='yes'>
      <pf dev='eth0'/>
    </forward>
  </network>

To use this network on the host, save the above code to a file, for example /tmp/passthrough.xml, and execute the following commands. Remember to replace eth0 with the real network interface name of your SR-IOV device's PF:

tux > sudo virsh net-define /tmp/passthrough.xml
tux > sudo virsh net-autostart passthrough
tux > sudo virsh net-start passthrough

14.9.4.2 Configuring VM Guests to Use VF from the Pool

The following example of VM Guest device interface definition uses a VF of the SR-IOV device from the pool created in Section 14.9.4.1, “Defining Network with Pool of VFs on VM Host Server”. libvirt automatically derives the list of all VFs associated with that PF the first time the guest is started.

<interface type='network'>
  <source network='passthrough'>
</interface>

After the first VM Guest starts that uses the network with the pool of VFs, verify the list of associated VFs. Do so by running virsh net-dumpxml passthrough on the host.

<network connections='1'>
  <name>passthrough</name>
  <uuid>a6a26429-d483-d4ed-3465-4436ac786437</uuid>
  <forward mode='hostdev' managed='yes'>
    <pf dev='eth0'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x1'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x3'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x5'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x10' function='0x7'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x11' function='0x1'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x11' function='0x3'/>
    <address type='pci' domain='0x0000' bus='0x02' slot='0x11' function='0x5'/>
  </forward>
  </network>

14.10 Configuring Storage Devices

Storage devices are defined within the disk element. The usual disk element supports several attributes. The following two attributes are the most important:

  • The type attribute describes the source of the virtual disk device. Valid values are file , block , dir , network , or volume .

  • The device attribute indicates how the disk is exposed to the VM Guest OS. As an example, possible values can include floppy , disk , cdrom , and others.

The following child elements are the most important:

  • driver contains the driver and the bus. These are used by the VM Guest to work with the new disk device.

  • The target element contains the device name under which the new disk is shown in the VM Guest. It also contains the optional bus attribute, which defines the type of bus on which the new disk should operate.

The following procedure shows how to add storage devices to the VM Guest:

  1. Edit the configuration for an existing VM Guest:

    tux > sudo virsh edit sles15
  2. Add a disk element inside the disk element together with the attributes type and device:

    <disk type='file' device='disk'>
  3. Specify a driver element and use the default values:

    <driver name='qemu' type='qcow2'/>
  4. Create a disk image, which will be used as a source for the new virtual disk device:

    tux > sudo qemu-img create -f qcow2 /var/lib/libvirt/images/sles15.qcow2 32G
  5. Add the path for the disk source:

    <source file='/var/lib/libvirt/images/sles15.qcow2'/>
  6. Define the target device name in the VM Guest and the bus on which the disk should work:

    <target dev='vda' bus='virtio'/>
  7. Restart your VM:

    tux > sudo virsh start sles15

Your new storage device should be available in the VM Guest OS.

14.11 Configuring Video Devices

When using the Virtual Machine Manager, only the Video device model can be defined. The amount of allocated VRAM or 2D/3D acceleration can only be changed in the XML configuration.

14.11.1 Changing the Amount of Allocated VRAM

  1. Edit the configuration for an existing VM Guest:

    tux > sudo virsh edit sles15
  2. Change the size of the allocated VRAM:

    <video>
    <model type='vga' vram='65535' heads='1'>
    ...
    </model>
    </video>
  3. Check if the amount of VRAM in the VM has changed by looking at the amount in the Virtual Machine Manager.

14.11.2 Changing the State of 2D/3D Acceleration

  1. Edit the configuration for an existing VM Guest:

    tux > sudo virsh edit sles15
  2. To enable/disable 2D/3D acceleration, change the value of accel3d and accel2d accordingly:

    <video>
    <acceleration accel3d='yes' accel2d='no'>
    ...
    </model>
    </video>
Tip
Tip: Enabling 2D/3D Acceleration

Only vbox video devices are capable of 2D/3D acceleration. You cannot enable it on other video devices.

14.12 Using Macvtap to Share VM Host Server Network Interfaces

Macvtap provides direct attachment of a VM Guest virtual interface to a host network interface. The macvtap-based interface extends the VM Host Server network interface and has its own MAC address on the same Ethernet segment. Typically, this is used to make both the VM Guest and the VM Host Server show up directly on the switch that the VM Host Server is connected to.

Note
Note: Macvtap Cannot Be Used with a Linux Bridge

Macvtap cannot be used with network interfaces already connected to a Linux bridge. Before attempting to create the macvtap interface, remove the interface from the bridge.

Note
Note: VM Guest to VM Host Server Communication with Macvtap

When using macvtap, a VM Guest can communicate with other VM Guests, and with other external hosts on the network. But it cannot communicate with the VM Host Server on which the VM Guest runs. This is the defined behavior of macvtap, because of the way the VM Host Server's physical Ethernet is attached to the macvtap bridge. Traffic from the VM Guest into that bridge that is forwarded to the physical interface cannot be bounced back up to the VM Host Server's IP stack. Similarly, traffic from the VM Host Server's IP stack that is sent to the physical interface cannot be bounced back up to the macvtap bridge for forwarding to the VM Guest.

Virtual network interfaces based on macvtap are supported by libvirt by specifying an interface type of direct. For example:

<interface type='direct'>
   <mac address='aa:bb:cc:dd:ee:ff'/>
   <source dev='eth0' mode='bridge'/>
   <model type='virtio'/>
   </interface>

The operation mode of the macvtap device can be controlled with the mode attribute. The following list show its possible values and a description for each:

  • vepa: All VM Guest packets are sent to an external bridge. Packets whose destination is a VM Guest on the same VM Host Server as where the packet originates from are sent back to the VM Host Server by the VEPA capable bridge (today's bridges are typically not VEPA capable).

  • bridge: Packets whose destination is on the same VM Host Server as where they originate from are directly delivered to the target macvtap device. Both origin and destination devices need to be in bridge mode for direct delivery. If either one of them is in vepa mode, a VEPA capable bridge is required.

  • private: All packets are sent to the external bridge and will only be delivered to a target VM Guest on the same VM Host Server if they are sent through an external router or gateway and that device sends them back to the VM Host Server. This procedure is followed if either the source or destination device is in private mode.

  • passthrough: A special mode that gives more power to the network interface. All packets will be forwarded to the interface, allowing virtio VM Guests to change the MAC address or set promiscuous mode to bridge the interface or create VLAN interfaces on top of it. Note that a network interface is not shareable in passthrough mode. Assigning an interface to a VM Guest will disconnect it from the VM Host Server. For this reason SR-IOV virtual functions are often assigned to the VM Guest in passthrough mode.

14.13 Disabling a Memory Balloon Device

Memory Balloon has become a default option for KVM. The device will be added to the VM Guest explicitly, so you do not need to add this element in the VM Guest's XML configuration. However, if you want to disable Memory Balloon in the VM Guest for any reason, you need to set model='none' as shown below:

<devices>
   <memballoon model='none'/>
</device>