Copy Linux sparse files over network

Wed, 09 Nov 2016 18:19:35 +0000

Sparse files are nice to use to store virtual machine’s virtual disks but can be a real pain in the ass to backup efficiently, especially over the network.
Luckily rsync provides a way to intelligently copy sparse files both locally and over the network.
The trick is use --sparse and --inplace options.
Let’s say we have a sparse 60 GB qemu virtual disk with only around 7 GB used:

$ ls -lh fedora24.qcow2 
-rw------- 1 root root 61G Nov 8 19:11 fedora24.qcow2
$ du -h fedora24.qcow2 
7.2G	fedora24.qcow2

The first thing to note is that ls does not recognize sparse files while du does.
The first time a file is copied use the --sparse option:

QEMU+KVM, reclaim disk space

Tue, 17 Nov 2015 11:06:29 +0000

After some time qcow2 images tend -especially after taking snapshots- to grow bigger and bigger, even bigger than the maximum size specified at creation time.
QEMU provides a tool called virt-sparsify (install libguestfs-tools package in CentOS 7) that can effectively make a virtual machine disk thin provisioned (space is not preallocated, only the actual space needed is used).
virt-sparsify has a nice number of options, the most interesting one is --in-place, it tells QEMU to shrink the volume in place without requiring any addition space.

Qcow2 on nagg.eu

Copy Linux sparse files over network

QEMU+KVM, reclaim disk space