About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Encrypted Paging
Abstract
With the z14 and APAR VM65993, z/VM 6.4 introduced encrypted paging. z/VM uses the Central Processor Assist for Cryptographic Function (CPACF) to accomplish this. One workload with various system configurations was used to evaluate the impact of encrypted paging.
On constant workload, as the cipher strength increased, the total CPU-busy spent on encryption or decryption increased. This was due to the increase in CPU time needed to encrypt a page.
On constant workload, the z14 with encrypted paging enabled performed better than a z13 did. Clients migrating from z13 to z14 can enable encrypted paging and still see performance improvement.
As the z/VM paging rate increased, the total CPU-busy spent on encryption or decryption increased. This was due to having to encrypt or decrypt more pages per second.
When a constant workload was run in larger and larger LPARs, the cost of encrypted paging did not change. The gross CPU cost of encrypted paging is a function of only the paging rate.
CPU time needed to decrypt a page is less than CPU time needed to encrypt a page. This suggests the CPU penalty for decrypting page reads will be less than the penalty for encrypting page writes.
The goal of encrypting pages at less than 5% increase in CPU/tx was achieved on our workloads. However, the exact increase any given workload experiences is a function of the CPU cost of a transaction and the fraction of transactions that incur paging. A light transaction with a heavy paging rate might make this goal unachievable. To ensure encrypted paging success, the transaction CPU cost and the paging rate should both be evaluated before enabling encrypted paging.
Introduction
This article provides a performance evaluation of select z/VM workloads running with encrypted paging enabled on the IBM z14.
Encrypted paging refers to the encryption of guest data by the Control Program as the data is paged out to paging volumes owned by z/VM. It constitutes an extra layer of protection for guest data. Encryption is limited to primary guest address space, data space, and VDISK pages. Spool files, directory pages, minidisk cache pages, CP page tables and minidisk data of a mapped minidisk pool will not be encrypted.
Encrypted paging takes advantage of the z14 Central Processor Assist for Cryptographic Function (CPACF) coprocessor present on each core.
Enabling Encryption
After the PTF is applied, encrypted paging is disabled by default. There are two ways to enable encrypted paging.
-
To enable
this function with the default cipher AES256 at
Initial Program Load (IPL) time,
include the following statement
in the z/VM system configuration file.
ENCRYPT PAGing ON For more information on the system configuration statement, see z/VM: CP Planning and Administration. - This support includes the ability to start or stop encrypted paging without incurring an IPL. For more information, see the SET ENCRYPT command at z/VM: CP Commands and Utilities Reference.
Cipher Strength
Encrypted paging supports Advanced Encryption Standard 128-bit (AES128), AES 192-bit (AES192), and AES 256-bit (AES256). The bit size of the key determines the strength of the encryption to be performed. By default, encrypted paging uses AES256. Once a cipher is set, changing it requires a z/VM IPL.
Encrypted Paging Metrics
New metrics are available in CP monitor record MRSTORSP, D3 R2. CP tracks the count of 4 KB pages being encrypted or decrypted and the CPU time consumed for this task. The metrics are extracted and formatted by the D3R2EC tool, available on the z/VM download page.
Method
IBM evaluated encrypted paging in a number of different scenarios. Each scenario was crafted to illustrate a specific aspect of the feature. For example, in one study a number of runs were done, varying only the cipher strength. The discussion below gives further detail.
All measurements used the Apache workload, the same z/VM CP load module, and the same Linux level.
IBM collected MONWRITE data during measurement steady state and reduced it with Performance Toolkit for VM. External throughput rate (ETR) was provided by the application. Encrypted paging metrics were reduced with the D3R2EC tool.
Results and Discussion
This section presents the results of the four evaluations.
Using Encryption on z14
Table 1 presents the impact of using encryption on a z14 workload, including the impacts of the different ciphers. This measurement was done on constant hardware, constant software, constant configuration, and constant paging rate.
| Table 1. Apache Paging Workload | ||||
| Run ID | A1TYA17E | A1TYA17I | A1TYA171 | A1TYA170 |
| Encrypted Paging | Disabled | Enabled | Enabled | Enabled |
| Cipher | na | AES128 | AES192 | AES256 |
| ETR | 1395.16 | 1410.35 | 1385.74 | 1398.56 |
| ITR | 1797.9 | 1752.0 | 1734.4 | 1737.4 |
| Total CPU/tx ratio (p) | 1.000 | 1.027 | 1.033 | 1.038 |
| Encrypted + Decrypted Paging Rate | na | 146163.09 | 145960.17 | 144873.30 |
| Pct CPU to Encrypt + Decrypt* | na | 16.35 | 17.09 | 18.07 |
| CPU time to Encrypt one page (usec) | na | 1.18 | 1.32 | 1.46 |
| CPU time to Decrypt one page (usec) | na | 1.06 | 1.02 | 1.02 |
|
Notes:
CEC model 3906-M05;
CP Assist for Cryptographic Function (CPACF) Support;
z/VM 6.4;
eight logical processors;
1 TB central storage;
Linux SLES12 SP1.
* 100% CPU = 1 IFL logical processor completely busy
| ||||
In the base case, encrypted paging was disabled. In the comparison measurements, the cipher strengths were AES128, AES192, and AES256, respectively. In spite of the high paging rate, all encrypted measurements met the goal of not increasing CPU consumption per transaction more than 5% above the encryption-disabled base case. CPU time consumed to encrypt one 4 KB page increased as the cipher strength increased. CPU time consumed to decrypt one 4 KB page decreased as the cipher strength increased.
z14 Encrypted Paging vs. z13
Table 2 presents the impact of encrypted paging on a workload that previously ran on a z13, where encrypted paging is not available. This comparison used constant software, constant configuration, constant workload, and constant paging rate.
| Table 2. Apache Paging Workload | ||||
| Run ID | A1TYA17Z | A1TYA170 | Delta | Pct Difference |
| Model | 2964-NC9 | 3906-M05 | ||
| Encrypted Paging | na | Enabled | ||
| Cipher | na | AES256 | ||
| ETR | 1316.58 | 1398.56 | 81.98 | 6.2 |
| ITR | 1481.0 | 1737.4 | 256.4 | 17.3 |
| Total CPU/tx | 5.77 | 5.15 | -0.62 | -10.7 |
| Encrypted + Decrypted Paging Rate | 0 | 144873.30 | 144873.30 | na |
| Pct CPU to Encrypt + Decrypt* | 0 | 18.02 | 18.02 | na |
|
Notes:
CP Assist for Cryptographic Function (CPACF) Support;
z/VM 6.4;
eight logical processors;
1 TB central storage;
Linux SLES12 SP1.
* 100% CPU = 1 IFL logical processor completely busy
| ||||
In the base case, the Apache workload was run on the z13. In the comparison run, the workload was run on the z14 with encrypted paging enabled with the default cipher setting AES256. Despite the extra cost of encryption, the workload performed better on the z14 with encrypted paging enabled.
Increasing the Paging Rate
Table 3 presents the additional CPU used for encryption or decryption as the paging rate increased to approximately 144,000 pages/second. This measurement was done on constant hardware, constant software, constant configuration, and constant cipher.
| Table 3. Apache Paging Scaling Workloads | ||||
| Run ID | A1TYA177 | A1TYA17D | A1TYA170 | |
| Light-paging | Medium-paging | Heavy-paging | ||
| Linux Server Count (p) | 100 | 110 | 128 | |
| ETR | 1955.96 | 1800.56 | 1398.56 | |
| ITR | 2022.7 | 1891.3 | 1737.4 | |
| Encrypted Paging Rate | 252.09 | 45384.10 | 74807.41 | |
| Decrypted Paging Rate | 57.68 | 35607.98 | 70065.89 | |
| Encrypted + Decrypted Paging Rate | 309.77 | 80986.08 | 144873.30 | |
| Pct CPU to Encrypt* | 0.038 | 6.673 | 10.891 | |
| Pct CPU to Decrypt* | 0.007 | 3.777 | 7.134 | |
| Pct CPU to Encrypt + Decrypt* | 0.045 | 10.451 | 18.026 | |
| CPU time to Encrypt one page (usec) | 1.53 | 1.47 | 1.46 | |
| CPU time to Decrypt one page (usec) | 1.24 | 1.06 | 1.02 | |
|
Notes:
CEC model 3906-M05;
CP Assist for Cryptographic Function (CPACF) Support;
z/VM 6.4;
eight logical processors;
1 TB central storage;
encrypted paging enabled with cipher AES256;
Linux SLES12 SP1.
* 100% CPU = 1 IFL logical processor completely busy
| ||||
To increase the paging rate, additional Linux servers were added to the workload to increase the paging rate while everything else remained constant. The light-paging workload used 100 Linux servers. The medium-paging workload used 110 Linux servers. The heavy-paging workload used 128 Linux servers. The CPACF facility can parallelize buffer decryption but it cannot parallelize buffer encryption. Therefore, decrypting a 4 KB page generally takes less CPU time than encrypting one.
Scaling Memory and Logical Processors
Table 4 presents encrypted paging on a heavy-paging Apache workload as the LPAR scaled memory and processors. This set of measurements was taken from the z/VM performance regression suite. The configuration was scaled from eight logical processors and 512 GB of central storage to 32 logical processors and 2 TB of central storage. This measurement was done on constant hardware, constant software, constant cipher, constant workload, and constant paging rate.
| Table 4. Apache Paging Workload | ||||
| Run ID | A05YA174 | A10YA172 | A15YA172 | A20YA172 |
| Logical Processors | 8 | 16 | 24 | 32 |
| Central Storage (GB) | 512 | 1024 | 1536 | 2048 |
| ETR | 1497.31 | 1430.89 | 1340.98 | 1307.06 |
| ITR | 2139.0 | 3230.0 | 4353.9 | 5166.3 |
| Encrypted + Decrypted Paging Rate | 159997.55 | 163467.08 | 150619.01 | 155924.89 |
| Pct CPU to Encrypt + Decrypt* | 19.57 | 19.57 | 17.96 | 18.88 |
|
Notes:
CEC model 3906-M05;
CP Assist for Cryptographic Function (CPACF) Support;
z/VM 6.4;
encrypted paging enabled with cipher AES256;
Linux SLES12 SP1.
* 100% CPU = 1 IFL logical processor completely busy
| ||||
In this set of measurements, the paging rate was held fairly constant. The percent of CPU used on encryption and decryption remained fairly constant as the LPAR scaled processors and central storage. This demonstrates that the CPU cost of encrypted paging is a function of the paging rate rather than of the LPAR size.
Summary
The encrypted paging performance goal is to encrypt paging at less than a 5% increase in CPU time per transaction. However, the exact increase any given workload experiences will be a function of both how much the workload is paging and how much CPU time a transaction consumes compared to how much CPU time the encryption consumes. If the transaction is very light and paging is very heavy, this goal might not be possible to reach. In this chapter, all workloads met the performance goals.
As the cipher strength increased, the percent of total CPU used on encryption and decryption increased. Additionally, CPU time used to encrypt a page increased and CPU time used to decrypt a page decreased.
As the z/VM paging rate increased, the percent of total CPU used on encryption and decryption increased.
On average, decryption costs less than encryption. The CPACF facility can parallelize buffer decryption but it cannot parallelize buffer encryption. This translates to the CPU penalty for page writes being greater than the CPU penalty for page reads.
Despite the extra cost of encryption, the z14 with encrypted paging enabled performed better when compared back to a z13.
The CPU cost of encrypted paging is a function of the paging rate rather than the size of the LPAR.
