Basic VM Guest management | Virtualization Guide

10.1 Listing VM Guests #Edit source

The VM Guest listing shows all VM Guests managed by libvirt on a VM Host Server.

10.1.1 Listing VM Guests with Virtual Machine Manager #Edit source

The main window of the Virtual Machine Manager lists all VM Guests for each VM Host Server it is connected to. Each VM Guest entry contains the machine's name, its status (Running, Paused, or Shutoff) displayed as an icon and literally, and a CPU usage bar.

10.1.2 Listing VM Guests with `virsh` #Edit source

Use the command virsh list to get a list of VM Guests:

List all running guests

> virsh list

List all running and inactive guests

> virsh list --all

For more information and further options, see virsh help list or man 1 virsh.

10.2 Accessing the VM Guest via console #Edit source

VM Guests can be accessed via a VNC connection (graphical console) or, if supported by the guest operating system, via a serial console.

10.2.1 Opening a graphical console #Edit source

Opening a graphical console to a VM Guest lets you interact with the machine like a physical host via a VNC connection. If accessing the VNC server requires authentication, you are prompted to enter a user name (if applicable) and a password.

When you click into the VNC console, the cursor is “grabbed” and cannot be used outside the console anymore. To release it, press Alt–Ctrl.

Tip: Seamless (absolute) cursor movement

To prevent the console from grabbing the cursor and to enable seamless cursor movement, add a tablet input device to the VM Guest. See Section 13.5, “Input devices” for more information.

Certain key combinations such as Ctrl–Alt–Del are interpreted by the host system and are not passed to the VM Guest. To pass such key combinations to a VM Guest, open the Send Key menu from the VNC window and choose the desired key combination entry. The Send Key menu is only available when using Virtual Machine Manager and virt-viewer. With Virtual Machine Manager, you can alternatively use the “sticky key” feature as explained in Tip: Passing key combinations to virtual machines.

Note: Supported VNC viewers

Principally all VNC viewers can connect to the console of a VM Guest. However, if you are using SASL authentication and/or TLS/SSL connection to access the guest, the options are limited. Common VNC viewers such as tightvnc or tigervnc support neither SASL authentication nor TLS/SSL. The only supported alternative to Virtual Machine Manager and virt-viewer is Remmina (refer to Book “Reference”, Chapter 4 “Remote graphical sessions with VNC”, Section 4.2 “Remmina: the remote desktop client”).

10.2.1.1 Opening a graphical console with Virtual Machine Manager #Edit source

In the Virtual Machine Manager, right-click a VM Guest entry.
Choose Open from the pop-up menu.

10.2.1.2 Opening a graphical console with `virt-viewer` #Edit source

virt-viewer is a simple VNC viewer with added functionality for displaying VM Guest consoles. For example, it can be started in “wait” mode, where it waits for a VM Guest to start before it connects. It also supports automatically reconnecting to a VM Guest that is rebooted.

virt-viewer addresses VM Guests by name, by ID or by UUID. Use virsh list --all to get this data.

To connect to a guest that is running or paused, use either the ID, UUID or name. VM Guests that are shut off do not have an ID—you can only connect to them by UUID or name.

Connect to guest with the ID 8

> virt-viewer 8

Connect to the inactive guest named sles12; the connection window opens once the guest starts

> virt-viewer --wait sles12

With the --wait option, the connection is upheld even if the VM Guest is not running at the moment. When the guest starts, the viewer is launched.

For more information, see virt-viewer --help or man 1 virt-viewer.

Note: Password input on remote connections with SSH

When using virt-viewer to open a connection to a remote host via SSH, the SSH password needs to be entered twice. The first time for authenticating with libvirt, the second time for authenticating with the VNC server. The second password needs to be provided on the command line where virt-viewer was started.

10.2.2 Opening a serial console #Edit source

Accessing the graphical console of a virtual machine requires a graphical environment on the client accessing the VM Guest. As an alternative, virtual machines managed with libvirt can also be accessed from the shell via the serial console and virsh. To open a serial console to a VM Guest named “sles12”, run the following command:

> virsh console sles12

virsh console takes two optional flags: --safe ensures exclusive access to the console, --force disconnects any existing sessions before connecting. Both features need to be supported by the guest operating system.

Being able to connect to a VM Guest via serial console requires that the guest operating system supports serial console access and is properly supported. Refer to the guest operating system manual for more information.

Tip: Enabling serial console access for SUSE Linux Enterprise and openSUSE guests

Serial console access in SUSE Linux Enterprise and openSUSE is disabled by default. To enable it, proceed as follows:

SLES 12, 15 and openSUSE

Launch the YaST Boot Loader module and switch to the Kernel Parameters tab. Add console=ttyS0 to the field Optional Kernel Command Line Parameter.

SLES 11

Launch the YaST Boot Loader module and select the boot entry for which to activate serial console access. Choose Edit and add console=ttyS0 to the field Optional Kernel Command Line Parameter. Additionally, edit /etc/inittab and uncomment the line with the following content:

#S0:12345:respawn:/sbin/agetty -L 9600 ttyS0 vt102

10.3 Changing a VM Guest's state: start, stop, pause #Edit source

Starting, stopping or pausing a VM Guest can be done with either Virtual Machine Manager or virsh. You can also configure a VM Guest to be automatically started when booting the VM Host Server.

When shutting down a VM Guest, you may either shut it down gracefully, or force the shutdown. The latter is equivalent to pulling the power plug on a physical host and is only recommended if there are no alternatives. Forcing a shutdown may cause file system corruption and loss of data on the VM Guest.

Tip: Graceful shutdown

To be able to perform a graceful shutdown, the VM Guest must be configured to support ACPI. If you have created the guest with the Virtual Machine Manager, ACPI should be available in the VM Guest.

Depending on the guest operating system, availability of ACPI may not be sufficient to perform a graceful shutdown. It is strongly recommended to test shutting down and rebooting a guest before using it in production. openSUSE or SUSE Linux Enterprise Desktop, for example, can require Polkit authorization for shutdown and reboot. Make sure this policy is turned off on all VM Guests.

If ACPI was enabled during a Windows XP/Windows Server 2003 guest installation, turning it on in the VM Guest configuration only is not sufficient. For more information, see:

Regardless of the VM Guest's configuration, a graceful shutdown is always possible from within the guest operating system.

10.3.1 Changing a VM Guest's state with Virtual Machine Manager #Edit source

Changing a VM Guest's state can be done either from Virtual Machine Manager's main window, or from a VNC window.

Procedure 10.1: State change from the Virtual Machine Manager window #

Right-click a VM Guest entry.
Choose Run, Pause, or one of the Shutdown options from the pop-up menu.

Procedure 10.2: State change from the VNC window #

Open a VNC Window as described in Section 10.2.1.1, “Opening a graphical console with Virtual Machine Manager”.
Choose Run, Pause, or one of the Shut Down options either from the toolbar or from the Virtual Machine menu.

10.3.1.1 Automatically starting a VM Guest #Edit source

You can automatically start a guest when the VM Host Server boots. This feature is not enabled by default and needs to be enabled for each VM Guest individually. There is no way to activate it globally.

Double-click the VM Guest entry in Virtual Machine Manager to open its console.
Choose View › Details to open the VM Guest configuration window.
Choose Boot Options and check Start virtual machine on host boot up.
Save the new configuration with Apply.

10.3.2 Changing a VM Guest's state with `virsh` #Edit source

In the following examples, the state of a VM Guest named “sles12” is changed.

Start

> virsh start sles12

Pause

> virsh suspend sles12

Resume (a suspended VM Guest)

> virsh resume sles12

Reboot

> virsh reboot sles12

Graceful shutdown

> virsh shutdown sles12

Force shutdown

> virsh destroy sles12

Turn on automatic start

> virsh autostart sles12

Turn off automatic start

> virsh autostart --disable sles12

10.4 Saving and restoring the state of a VM Guest #Edit source

Saving a VM Guest preserves the exact state of the guest’s memory. The operation is similar to hibernating a computer. A saved VM Guest can be quickly restored to its previously saved running condition.

When saved, the VM Guest is paused, its current memory state is saved to disk, and then the guest is stopped. The operation does not make a copy of any portion of the VM Guest’s virtual disk. The amount of time taken to save the virtual machine depends on the amount of memory allocated. When saved, a VM Guest’s memory is returned to the pool of memory available on the VM Host Server.

The restore operation loads a VM Guest’s previously saved memory state file and starts it. The guest is not booted but instead resumed at the point where it was previously saved. The operation is similar to coming out of hibernation.

The VM Guest is saved to a state file. Make sure there is enough space on the partition you are going to save to. For an estimation of the file size in megabytes to be expected, issue the following command on the guest:

> free -mh | awk '/^Mem:/ {print $3}'

Warning: Always restore saved guests

After using the save operation, do not boot or start the saved VM Guest. Doing so would cause the machine's virtual disk and the saved memory state to get out of synchronization. This can result in critical errors when restoring the guest.

To be able to work with a saved VM Guest again, use the restore operation. If you used virsh to save a VM Guest, you cannot restore it using Virtual Machine Manager. In this case, make sure to restore using virsh.

Important: Synchronize VM Guest's time after restoring it

If you restore the VM Guest after a long pause (hours) since it was saved, its time synchronization service—for example, chronyd—may refuse to synchronize its time. In this case, manually synchronize VM Guest's time. For example, for KVM hosts, you can use the QEMU guest agent and instruct the guest with the guest-set-time. Refer to Chapter 19, QEMU guest agent for more details.

Important: Only for VM Guests with disk types `raw`, `qcow2`

Saving and restoring VM Guests is only possible if the VM Guest is using a virtual disk of the type raw (.img), or qcow2.

10.4.1 Saving/restoring with Virtual Machine Manager #Edit source

Procedure 10.3: Saving a VM Guest #

Open a VNC connection window to a VM Guest. Make sure the guest is running.
Choose Virtual Machine › Shutdown › Save.

Procedure 10.4: Restoring a VM Guest #

Open a VNC connection window to a VM Guest. Make sure the guest is not running.
Choose Virtual Machine › Restore.
If the VM Guest was previously saved using Virtual Machine Manager, you are not offered an option to Run the guest. However, note the caveats on machines saved with virsh outlined in Warning: Always restore saved guests.

10.4.2 Saving and restoring with `virsh` #Edit source

Save a running VM Guest with the command virsh save and specify the file which it is saved to.

Save the guest named opensuse13

> virsh save opensuse13 /virtual/saves/opensuse13.vmsav

Save the guest with the ID 37

> virsh save 37 /virtual/saves/opensuse13.vmsave

To restore a VM Guest, use virsh restore:

> virsh restore /virtual/saves/opensuse13.vmsave

10.5 Creating and managing snapshots #Edit source

VM Guest snapshots are snapshots of the complete virtual machine including the state of CPU, RAM, devices and the content of all writable disks. To use virtual machine snapshots, all the attached hard disks need to use the qcow2 disk image format, and at least one of them needs to be writable.

Snapshots let you restore the state of the machine at a particular point in time. This is useful when undoing a faulty configuration or the installation of a lot of packages. After starting a snapshot that was created while the VM Guest was shut off, you need to boot it. Any changes written to the disk afterward are lost when starting the snapshot.

Note

Snapshots are supported on KVM VM Host Servers only.

10.5.1 Terminology #Edit source

There are several specific terms used to describe the types of snapshots:

Internal snapshots: Snapshots that are saved into the qcow2 file of the original VM Guest. The file holds both the saved state of the snapshot and the changes made since the snapshot was taken. The main advantage of internal snapshots is that they are all stored in one file and therefore it is easy to copy or move them across multiple machines.
External snapshots: When creating an external snapshot, the original qcow2 file is saved and made read-only, while a new qcow2 file is created to hold the changes. The original file is sometimes called a backing or base file, while the new file with all the changes is called an overlay or derived file. External snapshots are useful when performing backups of VM Guests. However, external snapshots are not supported by Virtual Machine Manager, and cannot be deleted by virsh directly. For more information on external snapshots in QEMU, refer to Section 33.2.4, “Manipulate disk images effectively”.
Live snapshots: Snapshots created when the original VM Guest is running. Internal live snapshots support saving the devices, and memory and disk states, while external live snapshots with virsh support saving either the memory state, or the disk state, or both.
Offline snapshots: Snapshots created from a VM Guest that is shut off. This ensures data integrity as all the guest's processes are stopped and no memory is in use.

10.5.2 Creating and managing snapshots with Virtual Machine Manager #Edit source

Important: Internal snapshots only

Virtual Machine Manager supports only internal snapshots, either live or offline.

To open the snapshot management view in Virtual Machine Manager, open the VNC window as described in Section 10.2.1.1, “Opening a graphical console with Virtual Machine Manager”. Now either choose View › Snapshots or click Manage VM Snapshots in the toolbar.

The list of existing snapshots for the chosen VM Guest is displayed in the left-hand part of the window. The snapshot that was last started is marked with a green tick. The right-hand part of the window shows details of the snapshot currently marked in the list. These details include the snapshot's title and time stamp, the state of the VM Guest at the time the snapshot was taken and a description. Snapshots of running guests also include a screenshot. The Description can be changed directly from this view. Other snapshot data cannot be changed.

10.5.2.1 Creating a snapshot #Edit source

To take a new snapshot of a VM Guest, proceed as follows:

Optionally, shut down the VM Guest to create an offline snapshot.
Click Add in the bottom left corner of the VNC window.
The window Create Snapshot opens.
Provide a Name and, optionally, a description. The name cannot be changed after the snapshot has been taken. To be able to identify the snapshot later easily, use a “speaking name”.
Confirm with Finish.

10.5.2.2 Deleting a snapshot #Edit source

To delete a snapshot of a VM Guest, proceed as follows:

Click Delete in the bottom left corner of the VNC window.
Confirm the deletion with Yes.

10.5.2.3 Starting a snapshot #Edit source

To start a snapshot, proceed as follows:

Click Run in the bottom left corner of the VNC window.
Confirm the start with Yes.

10.5.3 Creating and managing snapshots with `virsh` #Edit source

To list all existing snapshots for a domain (admin_server in the following), run the snapshot-list command:

> virsh snapshot-list --domain sle-ha-node1
 Name                 Creation Time             State
------------------------------------------------------------
 sleha_12_sp2_b2_two_node_cluster 2016-06-06 15:04:31 +0200 shutoff
 sleha_12_sp2_b3_two_node_cluster 2016-07-04 14:01:41 +0200 shutoff
 sleha_12_sp2_b4_two_node_cluster 2016-07-14 10:44:51 +0200 shutoff
 sleha_12_sp2_rc3_two_node_cluster 2016-10-10 09:40:12 +0200 shutoff
 sleha_12_sp2_gmc_two_node_cluster 2016-10-24 17:00:14 +0200 shutoff
 sleha_12_sp3_gm_two_node_cluster 2017-08-02 12:19:37 +0200 shutoff
 sleha_12_sp3_rc1_two_node_cluster 2017-06-13 13:34:19 +0200 shutoff
 sleha_12_sp3_rc2_two_node_cluster 2017-06-30 11:51:24 +0200 shutoff
 sleha_15_b6_two_node_cluster 2018-02-07 15:08:09 +0100 shutoff
 sleha_15_rc1_one-node 2018-03-09 16:32:38 +0100 shutoff

The snapshot that was last started is shown with the snapshot-current command:

> virsh snapshot-current --domain admin_server
Basic installation incl. SMT for CLOUD4

Details about a particular snapshot can be obtained by running the snapshot-info command:

> virsh snapshot-info --domain admin_server \
   -name  "Basic installation incl. SMT for CLOUD4"
Name:           Basic installation incl. SMT for CLOUD4
Domain:         admin_server
Current:        yes
State:          shutoff
Location:       internal
Parent:         Basic installation incl. SMT for CLOUD3-HA
Children:       0
Descendants:    0
Metadata:       yes

10.5.3.1 Creating internal snapshots #Edit source

To take an internal snapshot of a VM Guest, either a live or offline, use the snapshot-create-as command as follows:

> virsh snapshot-create-as --domain admin_server1 --name "Snapshot 1"2 \
--description "First snapshot"3

1	Domain name. Mandatory.
2	Name of the snapshot. It is recommended to use a “speaking name”, since that makes it easier to identify the snapshot. Mandatory.
3	Description for the snapshot. Optional.

10.5.3.2 Creating external snapshots #Edit source

With virsh, you can take external snapshots of the guest's memory state, disk state, or both.

To take both live and offline external snapshots of the guest's disk, specify the --disk-only option:

> virsh snapshot-create-as --domain admin_server --name \
 "Offline external snapshot" --disk-only

You can specify the --diskspec option to control how the external files are created:

> virsh snapshot-create-as --domain admin_server --name \
 "Offline external snapshot" \
 --disk-only --diskspec vda,snapshot=external,file=/path/to/snapshot_file

To take a live external snapshot of the guest's memory, specify the --live and --memspec options:

> virsh snapshot-create-as --domain admin_server --name \
 "Offline external snapshot" --live \
 --memspec snapshot=external,file=/path/to/snapshot_file

To take a live external snapshot of both the guest's disk and memory states, combine the --live, --diskspec, and --memspec options:

> virsh snapshot-create-as --domain admin_server --name \
 "Offline external snapshot" --live \
 --memspec snapshot=external,file=/path/to/snapshot_file
 --diskspec vda,snapshot=external,file=/path/to/snapshot_file

Refer to the SNAPSHOT COMMANDS section in man 1 virsh for more details.

10.5.3.3 Deleting a snapshot #Edit source

External snapshots cannot be deleted with virsh. To delete an internal snapshot of a VM Guest and restore the disk space it occupies, use the snapshot-delete command:

> virsh snapshot-delete --domain admin_server --snapshotname "Snapshot 2"

10.5.3.4 Starting a snapshot #Edit source

To start a snapshot, use the snapshot-revert command:

> virsh snapshot-revert --domain admin_server --snapshotname "Snapshot 1"

To start the current snapshot (the one the VM Guest was started off), it is sufficient to use --current rather than specifying the snapshot name:

> virsh snapshot-revert --domain admin_server --current

10.6 Deleting a VM Guest #Edit source

By default, deleting a VM Guest using virsh removes only its XML configuration. Since attached storage is not deleted by default, you can reuse it with another VM Guest. With Virtual Machine Manager, you can also delete a guest's storage files as well.

10.6.1 Deleting a VM Guest with Virtual Machine Manager #Edit source

In the Virtual Machine Manager, right-click a VM Guest entry.
From the context menu, choose Delete.
A confirmation window opens. Clicking Delete permanently erases the VM Guest. The deletion is not recoverable.
You can also permanently delete the guest's virtual disk by activating Delete Associated Storage Files. The deletion is not recoverable either.

10.6.2 Deleting a VM Guest with `virsh` #Edit source

To delete a VM Guest, it needs to be shut down first. It is not possible to delete a running guest. For information on shutting down, see Section 10.3, “Changing a VM Guest's state: start, stop, pause”.

To delete a VM Guest with virsh, run virsh undefine VM_NAME.

> virsh undefine sles12

There is no option to automatically delete the attached storage files. If they are managed by libvirt, delete them as described in Section 8.2.1.4, “Deleting volumes from a storage pool”.

10.7 Migrating VM Guests #Edit source

One of the major advantages of virtualization is that VM Guests are portable. When a VM Host Server needs to go down for maintenance, or when the host gets overloaded, the guests can easily be moved to another VM Host Server. KVM and Xen even support “live” migrations during which the VM Guest is constantly available.

10.7.1 Migration requirements #Edit source

To successfully migrate a VM Guest to another VM Host Server, the following requirements need to be met:

Important

To use the post-copy live migration option, KVM hosts with kernel version 5.11 or newer require setting the unprivileged_userfaultfd system value to 1:

> sudo sysctl -w vm.unprivileged_userfaultfd=1

Since kernel versions before 5.11 did not require setting unprivileged_userfaultfd to use the post-copy option, libvirt provides the setting in /usr/lib/sysctl.d/60-qemu-postcopy-migration.conf file to preserve the old behavior.

The source and destination systems must have the same architecture.
Storage devices must be accessible from both machines (for example, via NFS or iSCSI) and must be configured as a storage pool on both machines. For more information, see Chapter 12, Advanced storage topics.
This is also true for CD-ROM or floppy images that are connected during the move. However, you can disconnect them before the move as described in Section 13.11, “Ejecting and changing floppy or CD/DVD-ROM media with Virtual Machine Manager”.
libvirtd needs to run on both VM Host Servers and you must be able to open a remote libvirt connection between the target and the source host (or vice versa). Refer to Section 11.3, “Configuring remote connections” for details.
If a firewall is running on the target host, ports need to be opened to allow the migration. If you do not specify a port during the migration process, libvirt chooses one from the range 49152:49215. Make sure that either this range (recommended) or a dedicated port of your choice is opened in the firewall on the target host.
Host and target machine should be in the same subnet on the network, otherwise networking fails after the migration.
All VM Host Servers participating in migration must have the same UID for the qemu user and the same GIDs for the kvm, qemu, and libvirt groups.
No running or paused VM Guest with the same name must exist on the target host. If a shut down machine with the same name exists, its configuration is overwritten.
All CPU models except host cpu model are supported when migrating VM Guests.
SATA disk device type is not migratable.
File system pass-through feature is incompatible with migration.
The VM Host Server and VM Guest need to have proper timekeeping installed. See Chapter 17, VM Guest clock settings.
No physical devices can be passed from host to guest. Live migration is currently not supported when using devices with PCI pass-through or SR-IOV. If live migration needs to be supported, you need to use software virtualization (paravirtualization or full virtualization).
Cache mode setting is an important setting for migration. See: Section 16.6, “Cache modes and live migration”.
The image directory should be located in the same path on both hosts.
All hosts should be on the same level of microcode (especially the spectre microcode updates). This can be achieved by installing the latest updates of openSUSE Leap on all hosts.

10.7.2 Migrating with Virtual Machine Manager #Edit source

When using the Virtual Machine Manager to migrate VM Guests, it does not matter on which machine it is started. You can start Virtual Machine Manager on the source or the target host or even on a third host. In the latter case you need to be able to open remote connections to both the target and the source host.

Start Virtual Machine Manager and establish a connection to the target or the source host. If the Virtual Machine Manager was started neither on the target nor the source host, connections to both hosts need to be opened.
Right-click the VM Guest that you want to migrate and choose Migrate. Make sure the guest is running or paused—it is not possible to migrate guests that are shut down.
Tip: Increasing the speed of the migration
To increase the speed of the migration, pause the VM Guest. This is the equivalent of the former so-called “offline migration” option of Virtual Machine Manager.
Choose a New Host for the VM Guest. If the desired target host does not show up, make sure that you are connected to the host.
To change the default options for connecting to the remote host, under Connection, set the Mode, and the target host's Address (IP address or host name) and Port. If you specify a Port, you must also specify an Address.
Under Advanced options, choose whether the move should be permanent (default) or temporary, using Temporary move.
Additionally, there is the option Allow unsafe, which allows migrating without disabling the cache of the VM Host Server. This can speed up the migration but only works when the current configuration allows for a consistent view of the VM Guest storage without using cache="none"/0_DIRECT.
Note: Bandwidth option
In recent versions of Virtual Machine Manager, the option of setting a bandwidth for the migration has been removed. To set a specific bandwidth, use virsh instead.
To perform the migration, click Migrate.
When the migration is complete, the Migrate window closes and the VM Guest is now listed on the new host in the Virtual Machine Manager window. The original VM Guest is still available on the target host (in shut down state).

10.7.3 Migrating with `virsh` #Edit source

To migrate a VM Guest with virsh migrate, you need to have direct or remote shell access to the VM Host Server, because the command needs to be run on the host. The migration command looks like this:

> virsh migrate [OPTIONS] VM_ID_or_NAME CONNECTION_URI [--migrateuri tcp://REMOTE_HOST:PORT]

The most important options are listed below. See virsh help migrate for a full list.

--live: Does a live migration. If not specified, the guest is paused during the migration (“offline migration”).
--suspend: Does an offline migration and does not restart the VM Guest on the target host.
--persistent: By default a migrated VM Guest is migrated temporarily, so its configuration is automatically deleted on the target host if it is shut down. Use this switch to make the migration persistent.
--undefinesource: When specified, the VM Guest definition on the source host is deleted after a successful migration (however, virtual disks attached to this guest are not deleted).
--parallel --parallel-connections NUM_OF_CONNECTIONS: Parallel migration can be used to increase migration data throughput in cases where a single migration thread is not capable of saturating the network link between source and destination hosts. On hosts with 40 GB network interfaces, it may require four migration threads to saturate the link. With parallel migration, the time required to migrate large memory VMs can be significantly reduced.

The following examples use mercury.example.com as the source system and jupiter.example.com as the target system; the VM Guest's name is opensuse131 with Id 37.

Offline migration with default parameters

> virsh migrate 37 qemu+ssh://tux@jupiter.example.com/system

Transient live migration with default parameters

> virsh migrate --live opensuse131 qemu+ssh://tux@jupiter.example.com/system

Persistent live migration; delete VM definition on source

> virsh migrate --live --persistent --undefinesource 37 \
qemu+tls://tux@jupiter.example.com/system

Offline migration using port 49152

> virsh migrate opensuse131 qemu+ssh://tux@jupiter.example.com/system \
--migrateuri tcp://@jupiter.example.com:49152

Note: Transient compared to persistent migrations

By default virsh migrate creates a temporary (transient) copy of the VM Guest on the target host. A shut down version of the original guest description remains on the source host. A transient copy is deleted from the server after it is shut down.

To create a permanent copy of a guest on the target host, use the switch --persistent. A shut down version of the original guest description remains on the source host, too. Use the option --undefinesource together with --persistent for a “real” move where a permanent copy is created on the target host and the version on the source host is deleted.

It is not recommended to use --undefinesource without the --persistent option, since this results in the loss of both VM Guest definitions when the guest is shut down on the target host.

10.7.4 Step-by-step example #Edit source

10.7.4.1 Exporting the storage #Edit source

First you need to export the storage, to share the Guest image between host. This can be done by an NFS server. In the following example we want to share the /volume1/VM directory for all machines that are on the network 10.0.1.0/24. We are using a SUSE Linux Enterprise NFS server. As root user, edit the /etc/exports file and add:

/volume1/VM 10.0.1.0/24  (rw,sync,no_root_squash)

You need to restart the NFS server:

> sudo systemctl restart nfsserver
> sudo exportfs
/volume1/VM      10.0.1.0/24

10.7.4.2 Defining the pool on the target hosts #Edit source

On each host where you want to migrate the VM Guest, the pool must be defined to be able to access the volume (that contains the Guest image). Our NFS server IP address is 10.0.1.99, its share is the /volume1/VM directory, and we want to get it mounted in the /var/lib/libvirt/images/VM directory. The pool name is VM. To define this pool, create a VM.xml file with the following content:

<pool type='netfs'>
  <name>VM</name>
  <source>
    <host name='10.0.1.99'/>
    <dir path='/volume1/VM'/>
    <format type='auto'/>
  </source>
  <target>
    <path>/var/lib/libvirt/images/VM</path>
    <permissions>
      <mode>0755</mode>
      <owner>-1</owner>
      <group>-1</group>
    </permissions>
  </target>
  </pool>

Then load it into libvirt using the pool-define command:

# virsh pool-define VM.xml

An alternative way to define this pool is to use the virsh command:

# virsh pool-define-as VM --type netfs --source-host 10.0.1.99 \
     --source-path /volume1/VM --target /var/lib/libvirt/images/VM
Pool VM created

The following commands assume that you are in the interactive shell of virsh which can also be reached by using the command virsh without any arguments. Then the pool can be set to start automatically at host boot (autostart option):

virsh # pool-autostart VM
Pool VM marked as autostarted

To disable the autostart:

virsh # pool-autostart VM --disable
Pool VM unmarked as autostarted

Check if the pool is present:

virsh # pool-list --all
 Name                 State      Autostart
-------------------------------------------
 default              active     yes
 VM                   active     yes

virsh # pool-info VM
Name:           VM
UUID:           42efe1b3-7eaa-4e24-a06a-ba7c9ee29741
State:          running
Persistent:     yes
Autostart:      yes
Capacity:       2,68 TiB
Allocation:     2,38 TiB
Available:      306,05 GiB

Warning: Pool needs to exist on all target hosts

Remember: this pool must be defined on each host where you want to be able to migrate your VM Guest.

10.7.4.3 Creating the volume #Edit source

The pool has been defined—now we need a volume which contains the disk image:

virsh # vol-create-as VM sled12.qcow2 8G --format qcow2
Vol sled12.qcow2 created

The volume names shown are used later to install the guest with virt-install.

10.7.4.4 Creating the VM Guest #Edit source

Let's create a openSUSE Leap VM Guest with the virt-install command. The VM pool is specified with the --disk option, cache=none is recommended if you do not want to use the --unsafe option while doing the migration.

# virt-install --connect qemu:///system --virt-type kvm --name \
   sled12 --memory 1024 --disk vol=VM/sled12.qcow2,cache=none --cdrom \
   /mnt/install/ISO/SLE-12-Desktop-DVD-x86_64-Build0327-Media1.iso --graphics \
   vnc --os-variant sled12
Starting install...
Creating domain...

10.7.4.5 Migrate the VM Guest #Edit source

Everything is ready to do the migration now. Run the migrate command on the VM Host Server that is currently hosting the VM Guest, and choose the destination.

virsh # migrate --live sled12 --verbose qemu+ssh://IP/Hostname/system
Password:
Migration: [ 12 %]

10.8 Monitoring #Edit source

10.8.1 Monitoring with Virtual Machine Manager #Edit source

After starting Virtual Machine Manager and connecting to the VM Host Server, a CPU usage graph of all the running guests is displayed.

It is also possible to get information about disk and network usage with this tool, however, you must first activate this in Preferences:

Run virt-manager.
Select Edit › Preferences.
Change the tab from General to Polling.
Activate the check boxes for the kind of activity you want to see: Poll Disk I/O, Poll Network I/O, and Poll Memory stats.
If desired, also change the update interval using Update status every n seconds.
Close the Preferences dialog.
Activate the graphs that should be displayed under View › Graph.

Afterward, the disk and network statistics are also displayed in the main window of the Virtual Machine Manager.

More precise data is available from the VNC window. Open a VNC window as described in Section 10.2.1, “Opening a graphical console”. Choose Details from the toolbar or the View menu. The statistics are displayed from the Performance entry of the left-hand tree menu.

10.8.2 Monitoring with `virt-top` #Edit source

virt-top is a command line tool similar to the well-known process monitoring tool top. virt-top uses libvirt and therefore is capable of showing statistics for VM Guests running on different hypervisors. It is recommended to use virt-top instead of hypervisor-specific tools like xentop.

By default virt-top shows statistics for all running VM Guests. Among the data that is displayed is the percentage of memory used (%MEM) and CPU (%CPU) and the uptime of the guest (TIME). The data is updated regularly (every three seconds by default). The following shows the output on a VM Host Server with seven VM Guests, four of them inactive:

virt-top 13:40:19 - x86_64 8/8CPU 1283MHz 16067MB 7.6% 0.5%
7 domains, 3 active, 3 running, 0 sleeping, 0 paused, 4 inactive D:0 O:0 X:0
CPU: 6.1%  Mem: 3072 MB (3072 MB by guests)

   ID S RDRQ WRRQ RXBY TXBY %CPU %MEM    TIME   NAME
    7 R  123    1  18K  196  5.8  6.0   0:24.35 sled12_sp1
    6 R    1    0  18K    0  0.2  6.0   0:42.51 sles12_sp1
    5 R    0    0  18K    0  0.1  6.0  85:45.67 opensuse_leap
    -                                           (Ubuntu_1410)
    -                                           (debian_780)
    -                                           (fedora_21)
    -                                           (sles11sp3)

By default the output is sorted by ID. Use the following key combinations to change the sort field:

Shift–P: CPU usage

Shift–M: Total memory allocated by the guest

Shift–T: Time

Shift–I: ID

To use any other field for sorting, press Shift–F and select a field from the list. To toggle the sort order, use Shift–R.

virt-top also supports different views on the VM Guests data, which can be changed on-the-fly by pressing the following keys:

0: default view

1: show physical CPUs

2: show network interfaces

3: show virtual disks

virt-top supports more hot keys to change the view of the data and many command line switches that affect the behavior of the program. For more information, see man 1 virt-top.

10.8.3 Monitoring with `kvm_stat` #Edit source

kvm_stat can be used to trace KVM performance events. It monitors /sys/kernel/debug/kvm, so it needs the debugfs to be mounted. On openSUSE Leap it should be mounted by default. In case it is not mounted, use the following command:

> sudo mount -t debugfs none /sys/kernel/debug

kvm_stat can be used in three different modes:

kvm_stat                    # update in 1 second intervals
kvm_stat -1                 # 1 second snapshot
kvm_stat -l > kvmstats.log  # update in 1 second intervals in log format
                            # can be imported to a spreadsheet

Example 10.1: Typical output of `kvm_stat` #

kvm statistics

 efer_reload                  0       0
 exits                 11378946  218130
 fpu_reload               62144     152
 halt_exits              414866     100
 halt_wakeup             260358      50
 host_state_reload       539650     249
 hypercalls                   0       0
 insn_emulation         6227331  173067
 insn_emulation_fail          0       0
 invlpg                  227281      47
 io_exits                113148      18
 irq_exits               168474     127
 irq_injections          482804     123
 irq_window               51270      18
 largepages                   0       0
 mmio_exits                6925       0
 mmu_cache_miss           71820      19
 mmu_flooded              35420       9
 mmu_pde_zapped           64763      20
 mmu_pte_updated              0       0
 mmu_pte_write           213782      29
 mmu_recycled                 0       0
 mmu_shadow_zapped       128690      17
 mmu_unsync                  46      -1
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed               1553821     857
 pf_guest               1018832     562
 remote_tlb_flush        174007      37
 request_irq                  0       0
 signal_exits                 0       0
 tlb_flush               394182     148

See http://clalance.blogspot.com/2009/01/kvm-performance-tools.html for further information on how to interpret these values.

Virtualization Guide

10 Basic VM Guest management #Edit source

Note: Managing VM Guests on a remote VM Host Server

10.1 Listing VM Guests #Edit source

10.1.1 Listing VM Guests with Virtual Machine Manager #Edit source

10.1.2 Listing VM Guests with virsh #Edit source

10.2 Accessing the VM Guest via console #Edit source

10.2.1 Opening a graphical console #Edit source

Tip: Seamless (absolute) cursor movement

Note: Supported VNC viewers

10.2.1.1 Opening a graphical console with Virtual Machine Manager #Edit source

10.2.1.2 Opening a graphical console with virt-viewer #Edit source

Note: Password input on remote connections with SSH

10.2.2 Opening a serial console #Edit source

Tip: Enabling serial console access for SUSE Linux Enterprise and openSUSE guests

10.3 Changing a VM Guest's state: start, stop, pause #Edit source

Tip: Graceful shutdown

10.3.1 Changing a VM Guest's state with Virtual Machine Manager #Edit source

Procedure 10.1: State change from the Virtual Machine Manager window #

Procedure 10.2: State change from the VNC window #

10.3.1.1 Automatically starting a VM Guest #Edit source

10.3.2 Changing a VM Guest's state with virsh #Edit source

10.4 Saving and restoring the state of a VM Guest #Edit source

Warning: Always restore saved guests

Important: Synchronize VM Guest's time after restoring it

Important: Only for VM Guests with disk types raw, qcow2

10.4.1 Saving/restoring with Virtual Machine Manager #Edit source

Procedure 10.3: Saving a VM Guest #

Procedure 10.4: Restoring a VM Guest #

10.4.2 Saving and restoring with virsh #Edit source

10.5 Creating and managing snapshots #Edit source

Note

10.5.1 Terminology #Edit source

10.5.2 Creating and managing snapshots with Virtual Machine Manager #Edit source

Important: Internal snapshots only

10.5.2.1 Creating a snapshot #Edit source

10.5.2.2 Deleting a snapshot #Edit source

10.5.2.3 Starting a snapshot #Edit source

10.5.3 Creating and managing snapshots with virsh #Edit source

10.5.3.1 Creating internal snapshots #Edit source

10.5.3.2 Creating external snapshots #Edit source

10.5.3.3 Deleting a snapshot #Edit source

10.5.3.4 Starting a snapshot #Edit source

10.6 Deleting a VM Guest #Edit source

10.6.1 Deleting a VM Guest with Virtual Machine Manager #Edit source

10.6.2 Deleting a VM Guest with virsh #Edit source

10.7 Migrating VM Guests #Edit source

10.7.1 Migration requirements #Edit source

Important

10.7.2 Migrating with Virtual Machine Manager #Edit source

Tip: Increasing the speed of the migration

Note: Bandwidth option

10.7.3 Migrating with virsh #Edit source

Note: Transient compared to persistent migrations

10.7.4 Step-by-step example #Edit source

10.7.4.1 Exporting the storage #Edit source

10.7.4.2 Defining the pool on the target hosts #Edit source

Warning: Pool needs to exist on all target hosts

10.7.4.3 Creating the volume #Edit source

10.7.4.4 Creating the VM Guest #Edit source

10.7.4.5 Migrate the VM Guest #Edit source

10.8 Monitoring #Edit source

10.8.1 Monitoring with Virtual Machine Manager #Edit source

10.8.2 Monitoring with virt-top #Edit source

10.8.3 Monitoring with kvm_stat #Edit source

Example 10.1: Typical output of kvm_stat #

10.1.2 Listing VM Guests with `virsh` #Edit source

10.2.1.2 Opening a graphical console with `virt-viewer` #Edit source

10.3.2 Changing a VM Guest's state with `virsh` #Edit source

Important: Only for VM Guests with disk types `raw`, `qcow2`

10.4.2 Saving and restoring with `virsh` #Edit source

10.5.3 Creating and managing snapshots with `virsh` #Edit source

10.6.2 Deleting a VM Guest with `virsh` #Edit source

10.7.3 Migrating with `virsh` #Edit source

10.8.2 Monitoring with `virt-top` #Edit source

10.8.3 Monitoring with `kvm_stat` #Edit source

Example 10.1: Typical output of `kvm_stat` #