Where are RAM snapshots stored by libvirt?

cm08

11/13/22, 6:01 PM

My understanding is that when I run this command to create a snapshot of a running domain:

virsh snapshot-create-as myvm snapshot1

that not only are internal snapshots created in each disk image file, but that the RAM state is also saved somewhere. So if I later revert to that snapshot, everything in RAM will be restored to the time of the snapshot creation.

I see the snapshots listed within the qcow2 disk images, and I see the XML file that was created to describe the snapshot and the domain configuration, but where is the actual contents of the RAM stored? And is it "permanent" so if I reboot the host machine, for example, the saved RAM state of the domain will still be available if I choose to revert to that snapshot?

371

1 + 0

kvm-virtualization

virsh

Score:1

Server

shodanshok

11/13/22, 8:30 PM

The qcow2 format does provide permanent internal storage for the virtual machine memory. When snapshotting a running qcow2-backed virtual machines, its RAM state is dumped inside the very same qcow2 file used for storage.

From the official qcow2 page:

Each snapshot is described by a header:

  typedef struct QCowSnapshotHeader {
      /* header is 8 byte aligned */
      uint64_t l1_table_offset;

      uint32_t l1_size;
      uint16_t id_str_size;
      uint16_t name_size;

      uint32_t date_sec;
      uint32_t date_nsec;

      uint64_t vm_clock_nsec;

      uint32_t vm_state_size;
      uint32_t extra_data_size; /* for extension */
      /* extra data follows */
      /* id_str follows */
      /* name follows  */
  } QCowSnapshotHeader;

which further explains:

vm_state_size gives the size of the virtual machine state which was saved as part of this snapshot. The state is saved to the location of the original L1 table, directly after the image header.

On the other hand, what does happen when taking a snapshot of a raw image file (ie: not using qcow2)? Libvirt calls these kind of snapshot an external snapshot (compare it to the internal snapshot of a qcow2 file), because a new qcow2 file is generated and linked to the original (and now read-only) raw file. Libvirt then saves the virtual machine state (ie: its RAM content) in a specific file, generally inside a subdir of /var/lib/libvirt/. A metadata XML files is finally created to "glue" the dump file to the disk state saved in the overlay file.

Libvirt support for external snapshot is not very robust; for example (if things have not changed very recently) you can create an external snapshot with a single command but you need multiple complex command to delete/revert it. At the same time, for production workload RedHat officially recommend using external snapshots because they are somewhat more robust and faster than a long chain of internal snapshots (and they sell RHEV which hides the complexity of external snapshots away from the user).

0 + 0