For z/VM 6.3 and newer Releases

The System z processors allow one to configure processor storage (memory) as either central or expanded storage. Various enhancements in VM have changed the use of central and expanded storage.

Starting in z/VM 6.3, it is no longer recommended to configure any of the memory as expanded storage unless it is being attached to a virtual machine for testing or some unique usage. IBM suggests you configure all the processor memory as central storage.

Starting in z/VM 6.4, expanded storage is no longer supported at all, and therefore must be configurated as central storage (real memory).

Releases prior to z/VM 6.3

The System z processors allow one to configure processor storage (memory) as either central or expanded storage. Various enhancements in VM have changed the use of central and expanded storage.

VM/ESA 1.2.2 introduced a minidisk cache that did not need expanded storage; and z/VM 3.1.0 provided 64-bit support which removed the 31-bit architected limit of 2GB of central storage. Confusion has been added in that some other operating systems dropped support for expanded storage when they added their 64-bit support. All of this begs the question, "should I have expanded storage for VM?"

We still recommend having some expanded storage for VM. Here are a few thoughts on why.

  • While configuring some expanded storage may result in more paging, it often results in more consistent or better response time. The paging algorithms in VM evolved around having a hierachy of paging devices. Expanded storage is the high speed paging device and DASD the slower one where we do block paging. This means expanded storage can act as a buffer for more active users as they switch slightly between working sets. These more active users do not compete with users coming from a completely paged out scenario.
  • The central versus expanded issue is related to the different implementations of LRU algorithms used between stealing from central storage and expanded storage. In real storage, we basically just use a reference bit which gets reset fairly often. While in expanded storage, we have the luxury of having an exact timestamp of a blocks last use. (I have grossly over simplified the explanation, it would take someone like Storage Management expert Bill Holder to do it justice). This allows us to do a better job of selecting pages to page out to DASD.
  • Another thing to watch out for is that in an environment that pages to DASD, the potential exists for transactions (as determined by CP) to break up with the paging I/O. This could cause a real storage only configuration to look like the throughput rate is lower.
  • Also configure some expanded storage if it is needed for guest testing. OS/390, VM, and Linux can all use expanded storage.

What about 64-bit support? Doesn't that obsolete expanded storage?

No. The need for a hierarchy still exists even with z/VM.

The other consideration is that with z/VM 5.1.0 and earlier, all CP code, all CP control blocks, and most guest pages being referenced for CP processing (I/O, IUCV, etc.) need to be below 2GB. This can create contention for storage below 2GB. Such contention is often indicated by significant paging to DASD and a large number of pages available above 2GB (as seen by QUERY FRAMES command or your favorite performance tool). Systems with a 2GB constraint may benefit from having more of their storage configured as expanded storage. See Pre-z/VM 5.2.0 64-bit Considerations for more information. z/VM 5.2.0 has removed most of these restrictions and therefore contention for storage below 2GB is much less likely (see z/VM 5.2.0 64-bit Considerations).

Are there any exceptions to this rule?

Yes, there are always exceptions. If you do very little paging and are not constrained below 2GB, then there is little difference. Also, if you have a small number of large guests, that may also be an exception case.

Bottom Line Recommendation

If on z/VM 6.3, configure all processor memory as central storage.

In prior releases, determine if your system has a 2GB storage constraint (see above). If it does not, try configuring 25% of total storage as expanded storage. Many systems do not need more than 2GB of expanded storage regardless of the total storage available. If your system does have a 2GB storage constaint, configure as expanded storage all storage that is not being effectively used as central storage. The number of >2G page frames on the available list is an indication of how much storage should be reconfigured as expanded storage.

Return to Performance Tips