About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
z/VM TCP/IP Ethernet Mode
Abstract
In z/VM 5.4, the TCP/IP stack can operate an OSA-Express adapter in Ethernet mode (data link layer, aka layer 2, of the OSI model). When operating in Ethernet mode, the device is referenced by its Media Access Control (MAC) address instead of by its Internet Protocol (IP) address. Data is transported and delivered in Ethernet frames, providing the ability to handle protocol-independent traffic.
With this new function, the z/VM TCP/IP stack can operate a real dedicated OSA-Express adapter in Ethernet mode. More important, the new function also lets the z/VM TCP/IP stack use a virtual OSA NIC in Ethernet mode, coupled either to an Ethernet-mode guest LAN or to an Ethernet-mode VSWITCH.
We set up a z/VM TCP/IP stack with a virtual OSA NIC operating in Ethernet mode and coupled that virtual NIC to a VSWITCH running in Ethernet mode. Measurements comparing this configuration to the corresponding IP-mode setup show an increase in throughput from 0% to 13%, a decrease in CPU time from 0% to 7% for a low-utilization OSA card, and a decrease from 0% to 3% in a fully utilized OSA card.
Introduction
The new z/VM TCP/IP Ethernet mode support lets z/VM TCP/IP connect to an Ethernet-mode guest LAN or to an IPv4 or IPv6 Ethernet-mode virtual switch. Letting a z/VM TCP/IP stack connect to an Ethernet-mode virtual switch lets the stack participate in link aggregation configurations, thereby providing increased bandwidth and continuous network connectivity for the stack.
In z/VM 5.3 and earlier, z/VM TCP/IP operated an OSA-Express in only IP mode (except the virtual switch controller which already provided Ethernet mode). The new support lets a z/VM TCP/IP stack operate an OSA-Express in Ethernet mode, thereby letting z/VM TCP/IP connect to a physical LAN segment in Ethernet mode.
For more information on configuring the z/VM TCP/IP stack to communicate in Ethernet mode, see z/VM Connectivity.
Method
The measurements were performed using a TCP/IP stack connected to a virtual switch (client) on one LPAR communicating with a similar TCP/IP stack connected to a virtual switch (server) on another LPAR. The following figure shows the environment for the measurements referred to in this section.
Figure 1. z/VM TCP/IP Measurement Environment
|
A complete set of CMS Application Workload Modeler (AWM) runs were done for request-response (RR) with a maximum transmission unit (MTU) size of 1492 and streaming (STR) with MTU sizes of 1492 and 8992.
For each MTU and workload combination above, runs were done simulating 1, 10 and 50 client/server connections between the client and server across LPARs.
For each configuration, a base case run was done with both virtual switches (client and server) operating in IP mode. A comparison case run was then done with both virtual switches operating in Ethernet mode.
All measurements were done on a 2094-733 with four dedicated processors in each of the two LPARs using OSA-Express2 1 Gigabit Ethernet (GbE) cards. CP Monitor data was captured and reduced using Performance Toolkit for VM. The results shown are from the LPAR on the client side.
Results and Discussion
The following tables display the results of the measurements. Within each table the data is shown first for the z/VM TCP/IP stack communicating in IP mode followed by the data for the z/VM TCP/IP stack communicating in Ethernet mode. The bottom section of each table shows the percent difference between the IP-mode and Ethernet-mode results.
| Client/Server connections | 01 | 10 | 50 |
| (run ID) | (vvsn0102) | (vvsn1002) | (vvsn5002) |
| 5.4 IP mode | |||
| MB/sec | 62.5 | 63.6 | 60.5 |
| Total CPU msec/MB | 6.86 | 10.02 | 14.61 |
| Emul CPU msec/MB | 2.70 | 4.29 | 7.09 |
| CP CPU msec/MB | 4.16 | 5.72 | 7.52 |
| Approx. OSA card utilization | 53% | 54% | 51% |
| 5.4 Ethernet mode | |||
| MB/sec | 67.1 | 71.7 | 67.0 |
| Total CPU msec/MB | 6.78 | 9.79 | 14.55 |
| Emul CPU msec/MB | 2.71 | 4.35 | 7.18 |
| CP CPU msec/MB | 4.07 | 5.44 | 7.37 |
| Approx. OSA card utilization | 57% | 60% | 57% |
| % diff | |||
| MB/sec | 7% | 13% | 11% |
| Total CPU msec/MB | -1% | -2% | 0% |
| Emul CPU msec/MB | 0% | 1% | 1% |
| CP CPU msec/MB | -2% | -5% | -2% |
| 2094-733; z/VM 5.4 | |||
As seen in Table 1, running in Ethernet mode shows a modest improvement in throughput and a slight decrease in CPU time in the case where the OSA card is not fully utilized.
| Client/Server connections | 01 | 10 | 50 |
| (run ID) | (vvsj0102) | (vvsj1002) | (vvsj5002) |
| 5.4 IP mode | |||
| MB/sec | 116.4 | 118 | 117.8 |
| Total CPU msec/MB | 5.58 | 6.50 | 6.84 |
| Emul CPU msec/MB | 2.23 | 2.75 | 2.87 |
| CP CPU msec/MB | 3.35 | 3.75 | 3.97 |
| Approx. OSA card utilization | 99% | 100% | 100% |
| 5.4 Ethernet mode | |||
| MB/sec | 116.4 | 117.9 | 117.8 |
| Total CPU msec/MB | 5.58 | 6.29 | 6.62 |
| Emul CPU msec/MB | 2.35 | 2.76 | 2.87 |
| CP CPU msec/MB | 3.24 | 3.53 | 3.75 |
| Approx. OSA card utilization | 99% | 100% | 100% |
| % diff | |||
| MB/sec | 0% | 0% | 0% |
| Total CPU msec/MB | 0% | -3% | -3% |
| Emul CPU msec/MB | 5% | 0% | 0% |
| CP CPU msec/MB | -3% | -6% | -6% |
| 2094-733; z/VM 5.4 | |||
Table 2 shows throughput is the same because the OSA cards are fully utilized in both cases, however, when running in Ethernet mode there is a slight decrease in CPU time.
| Client/Server connections | 01 | 10 | 50 |
| (run ID) | (vvrn0102) | (vvrn1002) | (vvrn5002) |
| 5.4 IP mode | |||
| Tx/sec | 1511.7 | 4286.8 | 6356.4 |
| Total CPU msec/Tx | .22 | .19 | .21 |
| Emul CPU msec/Tx | .10 | .11 | .12 |
| CP CPU msec/Tx | .12 | .09 | .09 |
| 5.4 Ethernet mode | |||
| Tx/sec | 1552.8 | 4368.5 | 6481.4 |
| Total CPU msec/Tx | .23 | .19 | .19 |
| Emul CPU msec/Tx | .10 | .10 | .11 |
| CP CPU msec/Tx | .13 | .08 | .08 |
| % diff | |||
| Tx/sec | 3% | 2% | 2% |
| Total CPU msec/Tx | 1% | -4% | -7% |
| Emul CPU msec/Tx | -3% | -2% | -6% |
| CP CPU msec/Tx | 4% | -6% | -8% |
| 2094-733; z/VM 5.4 | |||
The RR runs in Table 3 show a small improvement in throughput along with a decrease in CPU time when using Ethernet mode.
Performance Toolkit for VM
Performance Toolkit for VM can be used to determine whether a virtual switch is communicating in Ethernet mode.
Here is an example of the Performance Toolkit VNIC screen (FCX269) which shows a virtual switch in Ethernet mode. Ethernet mode is sometimes referred to as 'Layer 2' while IP mode is referred to as 'Layer 3'. In the screen below Ethernet mode is indicated by the number '2' under the column labeled 'L', where 'L' represents 'Layer'. For emphasis we have highlighted these fields in the excerpt.
FCX269 Run 2008/07/19 09:00:21 VNIC
Virtual Network Device Activity
From 2008/07/19 08:52:37
To 2008/07/19 09:00:07
For 450 Secs 00:07:30 This is a performance report for GDLGPRF2
_____________________________________________________________________________________
____ . . . . . . . .
<--- Outbound/s ---> <--- Inbou
<--- LAN ID --> Adapter Base Vswitch V Bytes < Packets > Bytes <
Addr Owner Name Owner Addr Grpname S L T T_Byte T_Pack T_Disc R_Byte R_P
<< ----------------- System -------------- >> 209510 3173 .0 75571k 50
F000 SYSTEM CCBVSW1 TCPCB2 F000 ........ X 2 Q 209510 3173 .0 75571k 50
Here is an example of the Performance Toolkit GVNIC screen (FCX268) which shows a virtual switch in IP mode. IP mode is indicated by the number '3' under the column labeled 'Tranp' (which is short for 'Transport mode'). For emphasis we have highlighted these fields in the excerpt.
FCX268 Run 2008/07/17 12:13:03 GVNIC
General Virtual Network Device Description
From 2008/07/17 12:05:03
To 2008/07/17 12:13:03
For 480 Secs 00:08:00 This is a performance report for GDLGPRF2
____________________________________________________________________________________
____ . . . . . .
<--- LAN ID --> Adapter Base Vswitch V
Addr Owner Name Owner Addr Grpname S Tranp Type
F000 SYSTEM CCBVSW1 TCPCB2 F000 ........ X 3 QDIO
Summary and Conclusions
In the workloads we measured, z/VM TCP/IP running its virtual OSA-Express NIC in Ethernet mode provided data rate and OSA utilization improvements compared to running the virtual NIC in IP mode.
Based on this, and based on our previous Ethernet-mode evaluation and comparison, we believe similar results would be obtained for the case of z/VM TCP/IP running a real OSA-Express in Ethernet mode.
From a performance perspective, the workloads we ran revealed no "down side" to running the virtual NIC in Ethernet mode.
