Extended Diagnose X'44' Fast Path
The Apache workload was used to create a z/VM 5.2.0 workload for evaluation of the performance improvement described in the Performance Improvements section.
In addition to the measurements provided in this section, there is a comparison to z/VM 5.1.0 described in Apache Nonpaging where this improvement is one of the contributing factors.
Since there are so many other differences between z/VM 5.1.0 and z/VM 5.2.0, isolating the benefit of this improvement by comparing to a z/VM 5.1.0 base was not possible, so base measurements were on the same z/VM 5.2.0 system with the Diagnose X'44' (DIAG 44) fast path extensions removed.
Comparisons were made on both a 5-way and a 9-way to show that the benefit varies inversely with processor utilization. As utilization increases on the real processors, virtual processors are more likely to be in a real processor queue. If any virtual processor is queued when a DIAG 44 is issued, the fast path conditions are not met and thus it must use the normal path.
An extra 9-way measurement was completed with additional virtual processors to show that the DIAG 44 fast path percentage varies inversely with the number of virtual processors. As the number of virtual processors is increased, more are likely to be in a real processor queue. If any virtual processor is queued when a DIAG 44 is issued, the fast path conditions are not met and thus it must use the normal path. There is no evaluation of the overall benefit with the extra virtual processors.
These three comparisons demonstrate all the following expected results.
- An increase in the total virtual DIAG 44 rate
- A high percentage of the virtual DIAG 44s handled by the fast path
- Increased transaction rate
- Increased virtual microseconds (µsec) per transaction
- Decreased CP µsec per transaction
- Decreased CP system utilization
- Less benefit as the base becomes more processor constrained
- Lower fast path percentage as the number of virtual processors is increased
The following table contains the Apache workload parameter settings.
Apache parameters for DIAG 44 workloads
| Client virtual processors | 3 or 6 |
| Server virtual processors | 1 |
| Client virtual machines | 2 |
| Server virtual machines | 20 |
| Server virtual storage | 1024M |
| Client connections per server | 1 |
| Number of 1M Files | 2000 |
| Location of the file | Xstor MDC |
The following table contains results for the 9-way measurement.
9-way Apache DIAG 44 workload measurement data
| Fast Path Code Enabled | No | Yes |
| Run ID | EXPT5200 | EXPTD440 |
| Tx rate | 76.994 | 82.146 |
| DIAG 44/sec - Normal | 121035 | 8898 |
| DIAG 44/sec - Fast Path | 0 | 487087 |
| DIAG 44/sec - Total | 121035 | 495985 |
| Percent Fast Path | 0 | 98 |
| Total µsec/Tx (h) | 110252 | 103788 |
| CP µsec/Tx (h) | 51515 | 43869 |
| Emul µsec/Tx (h) | 58737 | 59917 |
| Total Util/Proc | 84.7 | 83.6 |
| System Util/Proc | 5.6 | 1.7 |
| 2064-116; 9 dedicated processors, 30G central storage, 8G expanded storage; z/VM 5.2.0; 40 connections; 3 client virtual processors | ||
This comparison demonstrates the DIAG 44 fast path extensions provided the basic expected benefits.
- Total virtual DIAG 44 rate increased by 309%
- 98% of the virtual DIAG 44s were handled by the fast path
- Transaction rate increased by 6.7%
- Virtual µsec per transaction increased by 2.0%
- CP µsec per transaction decreased by 14%
- CP system utilization decreased 69%
The following table contains results for the 5-way measurements.
5-way Apache DIAG 44 workload measurement data
| Fast Path Code Enabled | No | Yes |
| Run ID | EXPT5209 | EXPTD442 |
| Tx rate | 88.949 | 90.232 |
| DIAG 44/sec - Normal | 9056 | 3436 |
| DIAG 44/sec - Fast Path | 0 | 20525 |
| DIAG 44/sec - Total | 9056 | 23961 |
| Percent Fast Path | 0 | 85 |
| Total µsec/Tx | 59893 | 59070 |
| CP µsec/Tx | 22941 | 22646 |
| Emul µsec/Tx | 36952 | 36424 |
| Total Util/Proc | 92.8 | 92.3 |
| System Util/Proc | 0.8 | 0.6 |
| 2064-116; 5 dedicated processors, 30G central storage, 8G expanded storage; z/VM 5.2.0; 40 connections; 3 client virtual processors | ||
Overall processor utilization was 92% compared to 83% for the 9-way measurement. Basic improvements for this 5-way comparison were not as good as the 9-way, thus demonstrating less benefit as the base becomes more processor constrained. As utilization increases on the real processors, virtual processors are more likely to be in a real processor queue. If any virtual processor is queued when a DIAG 44 is issued, the fast path conditions are not met and thus it must use the normal path.
- Total virtual DIAG 44 rate increased by 164%
- 85% of the virtual DIAG 44s were handled by the fast path
- Transaction rate increased by 1.4%
- Virtual µsec per transaction increased by -1.4%
- CP µsec per transaction decreased by 1.3%
- System utilization decreased by 25%
The following table contains results for the 9-way measurements with 3 and 6 virtual processors.
9-way Apache DIAG 44 workload measurement data
| Client virtual processors | 3 | 6 |
| Run ID | EXPTD440 | EXPTD441 |
| Tx rate | 82.146 | 77.751 |
| DIAG 44/sec - Normal | 8898 | 31945 |
| DIAG 44/sec - Fast Path | 487087 | 401210 |
| DIAG 44/sec - Total | 495985 | 433155 |
| Percent Fast Path | 98 | 92 |
| Total µsec/Tx | 103788 | 117033 |
| CP µsec/Tx | 43869 | 51716 |
| Emul µsec/Tx | 59917 | 65317 |
| Total Util/Proc | 83.6 | 91.5 |
| System Util/Proc | 1.7 | 3.6 |
| Sched Spin Lock Rate | 13735 | 28739 |
| Sched Pct Spin Time | 0.373 | 1.234 |
| 2064-116; 9 dedicated processors, 30G central storage, 8G expanded storage; z/VM 5.2.0; 40 connections | ||
This comparison demonstrates lower fast path percentage as the number of virtual processors is increased. Percent of the virtual DIAG 44s handled by the fast path decreased from 98% to 92%. As the number of virtual processors are increased, more are likely to be in a real processor queue. If any virtual processor is queued when a DIAG 44 is issued, the fast path conditions are not met and thus it must use the normal path.
Overall results with 6 virtual processors were not as good as the results with 3 virtual processors. Transaction rate decreased by 4.8% and CP µsec per transaction increased by 12%. The increase in CP µsec per transaction is caused by a 259% increase in the normal-path DIAG 44 rate. Processing a normal-path DIAG 44 requires use of the scheduler lock and this caused the scheduler spin lock rate to increase by 109%. The percent of time spinning on the scheduler lock increased by 230%. This also accounts for the 111% increase in CP system utilization.