z/VM Virtual Switch
z/VM 4.4.0 has added a special type of Guest LAN, called a virtual switch, which is capable of bridging a z/VM Guest LAN (type QDIO) to an associated real LAN connected by an OSA-Express adapter. The virtual switch is designed to help eliminate the need for virtual machines acting as routers. Virtual routers consume valuable processor cycles to process incoming and outgoing packets, requiring additional copying of the data being transported. The virtual switch helps alleviate this problem by moving the data directly between the real network adapter and the target or originating guest data buffers.
This section summarizes measurement results that assess the performance of the virtual switch comparing it with a router stack. These measurements were done for Linux as well as for the VM TCP/IP stack.
Methodology: An internal tool was used to drive request-response (RR), connect-request-response (CRR) and streaming (S) workloads. The request-response workload consisted of the client sending 200 bytes to the server and the server responding with 1000 bytes. This interaction lasted for 200 seconds. The connect-request-response workload had the client connecting, sending 64 bytes to the server, the server responding with 8K and the client then disconnecting. This same sequence was repeated for 200 seconds. The streaming workload consisted of the client sending 20 bytes to the server and the server responding with 20MB. This sequence was repeated for 400 seconds.
A complete set of runs, consisting of 3 trials for each case, was done with the maximum transmission unit (MTU) set to 1492 and 8992.
The measurements were done on a 2064-109 with 3 dedicated processors in each LPAR used. Each LPAR had 1GB of central storage and 2GB expanded storage. CP monitor data was captured for one LPAR during the measurement and reduced using VMPRF.
 |
 |
- a VM stack communicating with a VM router which was attached to an OSA, the requests then went through another OSA card which was attached to a VM router in another LPAR and it communicated with a VM stack in that LPAR
- the same was done for a Linux stack communicating with a Linux router
- the VM router in environment 1 was replaced with a virtual switch
- the Linux router in environment 2 was replaced with a virtual switch
- a VM router was substituted for the Linux router (on both sides)
Results:
The following tables compare results when running Linux TCP/IP
communicating through a Linux router, a VM router and a virtual switch.
Absolute values are given for throughput and CPU time and then ratios
for comparison of a Linux router replaced with a virtual switch and also
for a VM router replaced with virtual switch.
All samples are from 50 client-server pairs.
In almost all cases the throughput increased and CPU time decreased
when a virtual switch was used.
Data is not included here for the VM stack to VM router and VM stack
to virtual switch measurements, but the results were similar. CPU
time went down and throughput went up for most measurements.
Table 1. Comparison - Linux Router/VM Router with VSwitch - Streaming
MTU Size 1492
| Linux to Linux Router | Linux to VM Router | Linux to VSwitch
|
|---|---|---|---|
MB/sec CPU msec/MB | 28.09 54.68 | 53.98 20.06 | 53.24 16.23 |
ratio MB/sec ratio CPU msec/MB | 1.00 1.00 |
| 1.90 0.30 |
ratio MB/sec ratio CPU msec/MB |
| 1.00 1.00 | 0.99 0.81 |
| Note: 2064-109; z/VM 4.4.0 with 64-bit CP; TCP/IP 440; Linux 2.4.19 31-bit | |||
Table 2. Comparison - Linux Router/VM Router with VSwitch - Streaming
MTU Size 8992
| Linux to Linux Router | Linux to VM Router | Linux to VSwitch
|
|---|---|---|---|
MB/sec CPU msec/MB | 78.34 18.88 | 71.94 12.93 | 108.59 10.22 |
ratio MB/sec ratio CPU msec/MB | 1.00 1.00 |
| 1.39 0.54 |
ratio MB/sec ratio CPU msec/MB |
| 1.00 1.00 | 1.51 0.79 |
| Note: 2064-109; z/VM 4.4.0 with 64-bit CP; TCP/IP 440; Linux 2.4.19 31-bit | |||
Table 3. Comparison - Linux Router/VM Router with VSwitch - CRR
MTU Size 1492
| Linux to Linux Router | Linux to VM Router | Linux to VSwitch
|
|---|---|---|---|
trans/sec CPU msec/trans | 1426.63 1.09 | 2778.33 0.60 | 2966.96 0.40 |
ratio trans/sec ratio CPU msec/trans | 1.00 1.00 |
| 2.08 0.37 |
ratio trans/sec ratio CPU msec/trans |
| 1.00 1.00 | 1.07 0.67 |
| Note: 2064-109; z/VM 4.4.0 with 64-bit CP; TCP/IP 440; Linux 2.4.19 31-bit | |||
Table 4. Comparison - Linux Router/VM Router with VSwitch - CRR
MTU Size 8992
| Linux to Linux Router | Linux to VM Router | Linux to VSwitch
|
|---|---|---|---|
trans/sec CPU msec/trans | 2220.97 0.73 | 4146.52 0.41 | 4187.93 0.29 |
ratio trans/sec ratio CPU msec/trans | 1.00 1.00 |
| 1.89 0.40 |
ratio trans/sec ratio CPU msec/trans |
| 1.00 1.00 | 1.01 0.71 |
| Note: 2064-109; z/VM 4.4.0 with 64-bit CP; TCP/IP 440; Linux 2.4.19 31-bit | |||
Table 5. Comparison - Linux Router/VM Router with VSwitch - RR
MTU Size 1492
| Linux to Linux Router | Linux to VM Router | Linux to VSwitch
|
|---|---|---|---|
trans/sec CPU msec/trans | 9915.42 0.15 | 13078.97 0.09 | 18064.12 0.06 |
ratio trans/sec ratio CPU msec/trans | 1.00 1.00 |
| 1.82 0.40 |
ratio trans/sec ratio CPU msec/trans |
| 1.00 1.00 | 1.38 0.67 |
| Note: 2064-109; z/VM 4.4.0 with 64-bit CP; TCP/IP 440; Linux 2.4.19 31-bit | |||
Table 6. Comparison - Linux Router/VM Router with VSwitch - RR
MTU Size 8992
| Linux to Linux Router | Linux to VM Router | Linux to VSwitch
|
|---|---|---|---|
trans/sec CPU msec/trans | 9942.71 0.15 | 12985.37 0.09 | 17970.93 0.07 |
ratio trans/sec ratio CPU msec/trans | 1.00 1.00 |
| 1.81 0.47 |
ratio trans/sec ratio CPU msec/trans |
| 1.00 1.00 | 1.38 0.78 |
| Note: 2064-109; z/VM 4.4.0 with 64-bit CP; TCP/IP 440; Linux 2.4.19 31-bit | |||
The following charts compare the same environments but are focused on CPU time spent. The tables show virtual and CP msec/MB (for streaming) or msec/trans (for CRR and RR) for both the client and router stacks. For the virtual switch case, the router stack is the virtual switch controller. Absolute values are shown. Notice that there is no virtual time for the router in the virtual switch case.
 |
When using a virtual switch, CP time is charged to the VM TCP/IP controller while handling interrupts (ie processing QDIO buffers). Once the datagrams have been queued to the receiving stack, and receive processing starts, CP time is charged to the receiving stack. The reverse is true for the send case. While CP is extracting segments from the stack's buffers, the time is charged to the sending stack. At this point the buffers are given to the OSA-Express card and no further time is charged.
 |
The results for MTU size of 8992 were similar to those shown for MTU of 1492 and are therefore not shown.
Summary: Because the amount of processing needed to handle the data being presented by/to the OSA-Express has been compressed, fewer processor cycles are needed to handle the workload. This means there is the potential for improved throughput in cases that were CPU-constrained.
There are no known cases where a virtual switch would not improve either CPU consumption, throughput or both. So, if you now have stacks communicating to an OSA-Express through a stack router, then you will benefit by converting that router to a virtual switch.