Contents | Previous | Next

Encrypted Paging

Abstract

With the z14 and APAR VM65993, z/VM 6.4 introduced encrypted paging. z/VM uses the Central Processor Assist for Cryptographic Function (CPACF) to accomplish this. One workload with various system configurations was used to evaluate the impact of encrypted paging.

On constant workload, as the cipher strength increased, the total CPU-busy spent on encryption or decryption increased. This was due to the increase in CPU time needed to encrypt a page.

On constant workload, the z14 with encrypted paging enabled performed better than a z13 did. Clients migrating from z13 to z14 can enable encrypted paging and still see performance improvement.

As the z/VM paging rate increased, the total CPU-busy spent on encryption or decryption increased. This was due to having to encrypt or decrypt more pages per second.

When a constant workload was run in larger and larger LPARs, the cost of encrypted paging did not change. The gross CPU cost of encrypted paging is a function of only the paging rate.

CPU time needed to decrypt a page is less than CPU time needed to encrypt a page. This suggests the CPU penalty for decrypting page reads will be less than the penalty for encrypting page writes.

The goal of encrypting pages at less than 5% increase in CPU/tx was achieved on our workloads. However, the exact increase any given workload experiences is a function of the CPU cost of a transaction and the fraction of transactions that incur paging. A light transaction with a heavy paging rate might make this goal unachievable. To ensure encrypted paging success, the transaction CPU cost and the paging rate should both be evaluated before enabling encrypted paging.

Introduction

This article provides a performance evaluation of select z/VM workloads running with encrypted paging enabled on the IBM z14.

Encrypted paging refers to the encryption of guest data by the Control Program as the data is paged out to paging volumes owned by z/VM. It constitutes an extra layer of protection for guest data. Encryption is limited to primary guest address space, data space, and VDISK pages. Spool files, directory pages, minidisk cache pages, CP page tables and minidisk data of a mapped minidisk pool will not be encrypted.

Encrypted paging takes advantage of the z14 Central Processor Assist for Cryptographic Function (CPACF) coprocessor present on each core.

Enabling Encryption

After the PTF is applied, encrypted paging is disabled by default. There are two ways to enable encrypted paging.

  • To enable this function with the default cipher AES256 at Initial Program Load (IPL) time, include the following statement in the z/VM system configuration file. ENCRYPT PAGing ON For more information on the system configuration statement, see z/VM: CP Planning and Administration.

  • This support includes the ability to start or stop encrypted paging without incurring an IPL. For more information, see the SET ENCRYPT command at z/VM: CP Commands and Utilities Reference.

Cipher Strength

Encrypted paging supports Advanced Encryption Standard 128-bit (AES128), AES 192-bit (AES192), and AES 256-bit (AES256). The bit size of the key determines the strength of the encryption to be performed. By default, encrypted paging uses AES256. Once a cipher is set, changing it requires a z/VM IPL.

Encrypted Paging Metrics

New metrics are available in CP monitor record MRSTORSP, D3 R2. CP tracks the count of 4 KB pages being encrypted or decrypted and the CPU time consumed for this task. The metrics are extracted and formatted by the D3R2EC tool, available on the z/VM download page.

Method

IBM evaluated encrypted paging in a number of different scenarios. Each scenario was crafted to illustrate a specific aspect of the feature. For example, in one study a number of runs were done, varying only the cipher strength. The discussion below gives further detail.

All measurements used the Apache workload, the same z/VM CP load module, and the same Linux level.

IBM collected MONWRITE data during measurement steady state and reduced it with Performance Toolkit for VM. External throughput rate (ETR) was provided by the application. Encrypted paging metrics were reduced with the D3R2EC tool.

Results and Discussion

This section presents the results of the four evaluations.

Using Encryption on z14

Table 1 presents the impact of using encryption on a z14 workload, including the impacts of the different ciphers. This measurement was done on constant hardware, constant software, constant configuration, and constant paging rate.

Table 1. Apache Paging Workload
Run ID A1TYA17E A1TYA17I A1TYA171 A1TYA170
Encrypted Paging Disabled Enabled Enabled Enabled
Cipher na AES128 AES192 AES256
ETR 1395.16 1410.35 1385.74 1398.56
ITR 1797.9 1752.0 1734.4 1737.4
Total CPU/tx ratio (p) 1.000 1.027 1.033 1.038
Encrypted + Decrypted Paging Rate na 146163.09 145960.17 144873.30
Pct CPU to Encrypt + Decrypt* na 16.35 17.09 18.07
CPU time to Encrypt one page (usec) na 1.18 1.32 1.46
CPU time to Decrypt one page (usec) na 1.06 1.02 1.02
Notes: CEC model 3906-M05; CP Assist for Cryptographic Function (CPACF) Support; z/VM 6.4; eight logical processors; 1 TB central storage; Linux SLES12 SP1. * 100% CPU = 1 IFL logical processor completely busy

In the base case, encrypted paging was disabled. In the comparison measurements, the cipher strengths were AES128, AES192, and AES256, respectively. In spite of the high paging rate, all encrypted measurements met the goal of not increasing CPU consumption per transaction more than 5% above the encryption-disabled base case. CPU time consumed to encrypt one 4 KB page increased as the cipher strength increased. CPU time consumed to decrypt one 4 KB page decreased as the cipher strength increased.

z14 Encrypted Paging vs. z13

Table 2 presents the impact of encrypted paging on a workload that previously ran on a z13, where encrypted paging is not available. This comparison used constant software, constant configuration, constant workload, and constant paging rate.

Table 2. Apache Paging Workload
Run ID A1TYA17Z A1TYA170 Delta Pct Difference
Model 2964-NC9 3906-M05
Encrypted Paging na Enabled
Cipher na AES256
ETR 1316.58 1398.56 81.98 6.2
ITR 1481.0 1737.4 256.4 17.3
Total CPU/tx 5.77 5.15 -0.62 -10.7
Encrypted + Decrypted Paging Rate 0 144873.30 144873.30 na
Pct CPU to Encrypt + Decrypt* 0 18.02 18.02 na
Notes: CP Assist for Cryptographic Function (CPACF) Support; z/VM 6.4; eight logical processors; 1 TB central storage; Linux SLES12 SP1. * 100% CPU = 1 IFL logical processor completely busy

In the base case, the Apache workload was run on the z13. In the comparison run, the workload was run on the z14 with encrypted paging enabled with the default cipher setting AES256. Despite the extra cost of encryption, the workload performed better on the z14 with encrypted paging enabled.

Increasing the Paging Rate

Table 3 presents the additional CPU used for encryption or decryption as the paging rate increased to approximately 144,000 pages/second. This measurement was done on constant hardware, constant software, constant configuration, and constant cipher.

Table 3. Apache Paging Scaling Workloads
Run ID A1TYA177 A1TYA17D A1TYA170
Light-paging Medium-paging Heavy-paging
Linux Server Count (p) 100 110 128
ETR 1955.96 1800.56 1398.56
ITR 2022.7 1891.3 1737.4
Encrypted Paging Rate 252.09 45384.10 74807.41
Decrypted Paging Rate 57.68 35607.98 70065.89
Encrypted + Decrypted Paging Rate 309.77 80986.08 144873.30
Pct CPU to Encrypt* 0.038 6.673 10.891
Pct CPU to Decrypt* 0.007 3.777 7.134
Pct CPU to Encrypt + Decrypt* 0.045 10.451 18.026
CPU time to Encrypt one page (usec) 1.53 1.47 1.46
CPU time to Decrypt one page (usec) 1.24 1.06 1.02
Notes: CEC model 3906-M05; CP Assist for Cryptographic Function (CPACF) Support; z/VM 6.4; eight logical processors; 1 TB central storage; encrypted paging enabled with cipher AES256; Linux SLES12 SP1. * 100% CPU = 1 IFL logical processor completely busy

To increase the paging rate, additional Linux servers were added to the workload to increase the paging rate while everything else remained constant. The light-paging workload used 100 Linux servers. The medium-paging workload used 110 Linux servers. The heavy-paging workload used 128 Linux servers. The CPACF facility can parallelize buffer decryption but it cannot parallelize buffer encryption. Therefore, decrypting a 4 KB page generally takes less CPU time than encrypting one.

Scaling Memory and Logical Processors

Table 4 presents encrypted paging on a heavy-paging Apache workload as the LPAR scaled memory and processors. This set of measurements was taken from the z/VM performance regression suite. The configuration was scaled from eight logical processors and 512 GB of central storage to 32 logical processors and 2 TB of central storage. This measurement was done on constant hardware, constant software, constant cipher, constant workload, and constant paging rate.

Table 4. Apache Paging Workload
Run ID A05YA174 A10YA172 A15YA172 A20YA172
Logical Processors 8 16 24 32
Central Storage (GB) 512 1024 1536 2048
ETR 1497.31 1430.89 1340.98 1307.06
ITR 2139.0 3230.0 4353.9 5166.3
Encrypted + Decrypted Paging Rate 159997.55 163467.08 150619.01 155924.89
Pct CPU to Encrypt + Decrypt* 19.57 19.57 17.96 18.88
Notes: CEC model 3906-M05; CP Assist for Cryptographic Function (CPACF) Support; z/VM 6.4; encrypted paging enabled with cipher AES256; Linux SLES12 SP1. * 100% CPU = 1 IFL logical processor completely busy

In this set of measurements, the paging rate was held fairly constant. The percent of CPU used on encryption and decryption remained fairly constant as the LPAR scaled processors and central storage. This demonstrates that the CPU cost of encrypted paging is a function of the paging rate rather than of the LPAR size.

Summary

The encrypted paging performance goal is to encrypt paging at less than a 5% increase in CPU time per transaction. However, the exact increase any given workload experiences will be a function of both how much the workload is paging and how much CPU time a transaction consumes compared to how much CPU time the encryption consumes. If the transaction is very light and paging is very heavy, this goal might not be possible to reach. In this chapter, all workloads met the performance goals.

As the cipher strength increased, the percent of total CPU used on encryption and decryption increased. Additionally, CPU time used to encrypt a page increased and CPU time used to decrypt a page decreased.

As the z/VM paging rate increased, the percent of total CPU used on encryption and decryption increased.

On average, decryption costs less than encryption. The CPACF facility can parallelize buffer decryption but it cannot parallelize buffer encryption. This translates to the CPU penalty for page writes being greater than the CPU penalty for page reads.

Despite the extra cost of encryption, the z14 with encrypted paging enabled performed better when compared back to a z13.

The CPU cost of encrypted paging is a function of the paging rate rather than the size of the LPAR.

Contents | Previous | Next