Should Linux Swap to a Virtual Disk In Storage (VDISK)?

To explore swapping to VDISK, first we need to look at what happens when your Linux guest swaps. Next we examine administrative characteristics of VDISK. Finally, we look at considerations for storage-rich and storage-constrained systems.

What Happens When Your Linux Guest Swaps

When a Linux guest swaps, it is doing I/O to its swap DASD in support of switching among processes. Processes not running are moved to the swap device while those running remain in guest storage. This means:

• The Linux guest is incurring guest path length and I/O latency because it is doing swap I/O.
• z/VM is incurring Control Program path length and I/O latency on behalf of the guest when it does the I/O to the swap DASD on behalf of the guest.

Administrative Aspects of VDISK

A VDISK makes a very attractive swap device from an administration point of view. It is easy to set up a VDISK, in fact, usually your z/VM system administrator does not need to be involved. The CP DEFINE command or a user directory statement can be employed to set up a VDISK.

Performance Effects of Swapping to VDISK

Swapping to VDISK has potential for improving system performance. It can also be detrimental to performance. Following are the considerations one must keep in mind.

Storage-Rich Systems

If your system has plenty of real storage, you can reduce latency in Linux swap I/O by putting the Linux swap volume on a VDISK. The VDISK is a very fast I/O device when its disk blocks, implemented as pages in an address space, can remain memory-resident.

For VDISK I/O, z/VM does not pull the Linux guest's swap file I/O buffers to below the real 2 GB line in order to do the I/O for the guest. If you are storage-constrained below 2 GB but rich above, VDISK might be appropriate for you if not defined unnecessarily large.

Note that using a VDISK does not change the other I/O behaviors of Linux or z/VM. Linux must still form and emit I/Os to read and write the swap extent, and z/VM must still perform these I/Os for the guest.

Storage-Constrained Systems

Prior to z/VM 4.4.0, when a page fault occurs on a page of a VDISK, that page, and any other associated page for the fault, is brought into real memory below the 2 GB address. This may add to the contention for storage below 2 GB on those systems. The page fault behavior was changed in z/VM 4.4.0 so that the page now is brought in above 2 GB where possible (as is done with ordinary guest pages).

Though the pages backing the VDISK blocks themselves are not created until referenced, the architected control blocks for dynamic address translation (page tables and segment tables, mainly) are created at the time the VDISK is created. Further, these control blocks are not pageable and must reside below the 2 GB address in real memory. This is another factor that can increase contention for storage below 2 GB.

The VM storage management steal processing has a hierarchy of pages that it uses in trying to select the most appropriate pages. In that hierarchy, normal idle guest pages are chosen for steal more readily than VDISK pages (a virtual disk in storage page is really a system utility space and therefore is given preferential treatment). As Linux guests go idle, this hierarchy might have an undesireable effect. Linux will determine an unused page and move it out to its swap disk (a VDISK). If the Linux guest goes idle, VM storage management will steal more aggressively from the guest pages (pages Linux decided it needed) and less aggressively from the VDISK (pages Linux decided it was safe to page out). This is counter to what good performance would dictate.

Again we mention that for VDISK I/O, z/VM does not pull the Linux guest's swap file I/O buffers to below the real 2 GB line in order to do the I/O for the guest. If you are storage-constrained below 2 GB but rich above, VDISK might be particularly appropriate for you if not defined unnecessarily large.

You can tell whether you are storage-constrained by using VM PRF or Performance Toolkit, as illustrated in the table below.