Dynamic I/O on z/VM for Dummies®

I wrote this article because I was extremely confused as to how dynamic I/O works and what things I had to do. The information was available, but it was kind of spread out in obscure locations, with the diamonds barely visible in the sparkling grains of sand.

Dynamic I/O is a function of the z Systems CPC that enables a logical partition (LPAR) to programmatically alter the CPC I/O configuration. Since it is at the CPC level, those alterations can affect any or all LPARs on the CPC.

Unlike some other hardware platforms, a z Systems CPC works from a pre-defined I/O configuration that contains information about the

• Channel paths
• Control units that those paths can talk to
• Devices attached to those control units
• Message processors
• Message devices
• I/O adapters
• Logical partitions

All devices that the partitions need to use must be defined in the current I/O configuration. If it isn't in the current I/O configuration, it's not usable, even if it's physically plugged into the CPC. This helps to protect the integrity of the systems and their data.

If that seems a bit paranoid, it's because this isn't IBM's first rodeo. In some ways, that's what mainframes are all about and why they are the darlings of Hollywood: institutionalized paranoia. The value of the data on your mainframe is HUGE. If that means the hardware admins and the technicians plugging things in have to talk to each other, well, it's not the end of the world.

This tutorial is not a replacement for the product publications. I'm just going to try to teach you enough so that when you use the pubs, they'll make more sense. Got it? Good.

"Shall we begin, Professor?"

Terminology

There are some terms you're going to need to understand so that you can grok how devices are configured and how that configuration can be changed.

Don't worry, we'll cover them all in more detail. I'm going to start from the very beginning ("A very good place to start!") and look at how a z Systems server prepares itself to do battle with the outside world. It may seem like overkill, but it's good to know on those days when things go awry.

There are three views of the I/O configuration.

• The initial configuration: The I/O configuration in effect immediately after a Power-On Reset operation.

• The current configuration: The I/O configuration that is used by the channel subsystem to control I/O operations. If may or may not match the initial configuration. It depends on whether or not any dynamic I/O changes have been made. This may also be referred to as the active I/O configuration.

• The target configuration: An I/O configuration that will replace the current I/O configuration when a sequence of one or more dynamic configuration changes are successfully made.

These views will show up repeatedly in this tutorial.

Power-On Reset

In the Beginning, the machine is dark and cold. Silent and still. Waiting. Lo! A spark! Energy begins to course through its veins. Behold! Someone has turned on the power!

At this point the machine does a lot of stuff that neither you nor I would understand. But the configuration is validated, the boilers are lit, and the "OPEN" sign is placed in the front window. Think of it as the the Initial Microcode Load (IML) process on steroids. (When you turn on the TV, do you ask for details? I think not.)

But before the Welcome mat is rolled out, it's like a newborn baby, curious about its surroundings. It needs to know two things:

1. Which stored (saved) I/O configuration should be selected as the initial I/O configuration,
2. Which logical partitions defined in the selected I/O configuration should be activated.

The answers to both of those questions are kept in the CPC's reset activation profile. The reset profile is found by selecting the CPC in the Customize/Delete Activation Profile task.

With that information the machine can come up and you can get some work done! Yipee!!

The I/O Configuration Dataset (IOCDS)

The four possible saved I/O configurations are maintained on the CPC Support Element in an I/O configuration dataset (IOCDS) slot named A0, A1, A2, or A3. These are collectively referred to as Ax, and "slot" is just another name for a file.

There are three ways to populate an IOCDS:

1. The I/O Configuration Program (IOCP). This application takes human-readable text (in the form of assembler macros) from a file, compiles it into machine-readable form, and writes it to an IOCDS. IOCP is available on z/VM, z/OS, and z/VSE, and is described in the Input/Output Configuration Program User's Guide for ICP IOCP, SB10-7136.

2. Standalone IOCP (SA-IOCP). This program is built into the CPC. It includes both a file editor and the compiler. Is most often used to prepare the machine for its first operating system, but that's necessary only if you don't have access to another z Systems server that can be used to create the I/O configuration for you. Who knows, maybe you can get a friend at another company to help you. Or even to IBM.

   The data format used for SA-IOCP and IOCP is identical, so moving from one to the other is straightforward. Data can be imported from a memory stick, if desired. The program is described in the Stand-alone Input/Output Configuration Program User's Guide, SB10-7166.

3. The Hardware Configuration Definition (HCD) program. This is an application on z/VM and z/OS that not only performs performs the functions of IOCP, but also helps you create the input file. It understands all the rules, machine limits, and keywords, so it doesn't just create the input file, but creates one that's correct. A fill-in-the-blank "editor" and IOCP program rolled into one.

   To get started, you can import your IOCP source file. If you have the High-Level Assembler, you can do it yourself. If don't have it, open a z/VM PMR and ask IBM to do it for you. A complete description of migrating to HCD is in Chapter 15 of the z/VM I/O Configuration manual manual.

   Its companion program, Hardware Configuration Manager (HCM), provides a graphical user interface. You can read about it in the z/OS and z/VM Hardware Configuration Manager User's Guide, SC34-2670.

   Unlike z/OS, the z/VM version of HCD does NOT include a full-screen 3270 interface. If you don't have z/OS anywhere, then you will be using HCM.

Regardless of choice, changing the I/O configuration is a fundamental CPC management task. All partitions depend on it, so measure twice and cut once! A second pair of eyes looking at your proposed changes wouldn't go amiss, either.

Why are there four IOCDS slots? So that you can manage change. You change the next slot each time you make a change, letting you back out a change. You could even choose to reserve one of the slots for an emergency I/O configuration - one that doesn't necessarily obey all of your I/O partitioning rules. It's up to you.

As mentioned above, the reset activation profile determines which IOCDS will be loaded during a POR. It can explicitly load Ax or it can load whichever IOCDS is identified as active. The choice is made on the General tab of the Customize/Delete Activation Profile task.

HCD can also select the active IOCDS. In HCD, the active IOCDS is marked "POR". The others are marked "Alternate".

← See that D0? It can never be the active IOCDS.

You select it manually, but only when none of the others work, or when you're creating the first I/O configuration for a new CPC (or one that has only CFs) using the standalone version of IOCP.

No need to remember this; the SA-IOCP manual tells you when to use D0.

The Active IOCDS

The highlighted Use active IOCDS option is preferred because it fits much better into the dynamic I/O process, while still giving you a way to back out a change (more on that later).

But what is an "active" IOCDS? It's the IOCDS that the administrator has flagged using the the HCD CBDSACT or CP SET IOCDS_ACTIVE commands, or the Support Element. No more, no less. It has nothing to do with the current I/O configuration of the machine.

I mentioned above how you might have an "emergency" I/O configuration. It will not be in your active IOCDS rotation. When you need to use it, you have two choices. You can either change the reset activation profile to point to it, or you can set it as the active IOCDS. (In either case performing a POR afterwards.)

Here is how you use the I/O Configuration task on the Support Element to select an alternate IOCDS to be the active IOCDS:

IOCDS Security

Keep in mind that the ability to modify the I/O configuration means that someone could enable two LPARs to talk to each other or otherwise share data in a way that is undetectable from outside the machine. It is a privileged function that should allowed only on trusted LPARs with trusted admins.

Typically, the active IOCDS is locked (write-protected), and all other IOCDSes are unlocked. Whenever you make make an IOCDS active, that IOCDS will be locked and the previously-active IOCDS is unlocked.

Should the need arise, you can manually lock and unlock an IOCDS:

Details about the IOCDS write-protect attribute can be found in the Input/Output Configuration task on the Support Element.

IOCDS Partition List

There is even a way for you to plan ahead, adding unnamed LPARs to the IOCDS so that they can be assigned a name later, when needed. (If only the various books contained this paragraph!) Planning ahead is only required if you run IOCP yourself. When you use HCD, it takes care of this for you, so that you can create add new LPARs at any time witihout a POR.

The Partitions tab of the reset activation profile will show you the list of partitions to be started and their order of activation:

The Hardware System Area (HSA)

At POR the CPC loads the desired IOCDS into a special part of the machine called the Hardware System Area (HSA). This is where the current I/O configuration is stored, and it is from here that the channel subsystem obtains the details on how I/O is to be performed. This means that if a device isn't in the HSA, then the device doesn't exist as far as the channel subsystem is concerned.

A dynamic I/O configuration change is really just a change to the contents of the HSA. When you do that, each affected LPAR is notified of the change. It's up to you to tell the OS how to react to the change. More on that later.

Modifying the HSA

There are two ways to change the HSA.

1. HCD. Not only does HCD manage the IOCDS, it also manages the content of the HSA. It takes the "current" and "after" I/O configuration images, holds them up to the light, so to speak, and changes the current I/O configuration (in the HSA) to match.

2. CP commands. There's nothing magic about altering the HSA. You add, change, or delete, PCHIDs, CHIPDs, control units, devices, and partition identifiers (including LPAR names).

   The challenge here is to keep track of every change you make, and the order they were made, so that you can back them out if needed.

   If you do things wrong, you can end up with errors that are difficult to diagnose, or, worst case, an I/O configuration that can't be undone without a POR. It feels very much like the I/O subsystem wasn't really designed to be changed by people who don't know precisely what they're doing and how to do it.

When HCD is being used, the CP commands are locked out, and vice versa.

The Token

This is the mainframe, and we care deeply about system integrity. Because you can authorize more than one LPAR to make I/O configuration changes, it's incredibly important that the two LPARs not be permitted to step on each other.

When a dynamic I/O-enabled IOCDS is written, whether by IOCP (with the DYN option) or by HCD, a magic cookie called the token is added to it. When a POR is performed with that IOCDS and dynamic I/O is enabled on the CPC, the token is loaded into the channel subsystem.

The token serves two purposes. First, it's mere presence is an indication to the host software that the I/O configuration is capable of being changed dynamically. Second, it acts as a gatekeeper to ensure two processes don't try to change I/O configuration at the same time. Each time you successfully change the I/O configuration, you set a new token. If two concurrent processes start with the current token, the first one to perform the I/O change will succeed. The second will always fail because its "current" token will not be valid.

No token? Wrong token? In either case the I/O configuration cannot be changed dynamically.

The token comes in two flavors: VM and HCD.

When you create an I/O configuration with the IOCP program on z/VM, a VM token will be added. The only way to alter the I/O configuration is to turn on CP control (SET DYNAMIC_IO ON) and use the CP commands (DEFINE CHPID, MODIFY CU, etc.). CP will not modify an I/O configuration that doesn't have a VM token. In an odd stroke of luck, the token contains the string VM-TOKEN, so it's easy to identify via the CP QUERY TOKEN command:

     VM-TOKEN12/17/1509:52:22    ....12/17/1508:14:05

When you use HCD to create the I/O configuration, HCD uses a token of its own design:

     M28     ....S..... ....^........15-12-0115:25:40CBD191  IODF36

The CPC name, M28, is there, along with the IODF name (IODF36). Since it's not a VM token, you can't activate it using CP commands; you must use HCD.

Obviously there's no standard for token construction. It's just a way for CP and HCD to determine who created the I/O configuration - to ensure that there's only one cook in the kitchen.

You can also view the tokens on the Support Element (not HMC!) via the CPC Configuration "Input/Output (I/O) Configuration" task.

Below is IOCDS A0. Note the difference between the token contained in the IOCDS and the one in the HSA. Does that mean someone made a dynamic I/O change? No. Look at the timestamps in the tokens. The one in the IOCDS is later than the one in the HSA. All it means is that the IOCDS has been replaced at least once since the HSA was last loaded or changed.

It's what I would expect of someone who isn't using dynamic I/O. They write a new IOCDS and then POR, bringing the tokens back into sync. In fact, you can't enable dynamic I/O as long those two tokens are different.

Dynamic I/O Changes

There's nothing evil or intrinsically wrong about a POR, but it is disruptive. Using dynamic I/O configuration updates elminates the POR requirement. There are few one-time changes you need to make, but once that's done, POR will be for disruptive maintenance only.

Even though the HSA is loaded from an IOCDS, there no built-in management connection between the two. Which is to say, modifying an IOCDS doesn't change the HSA, and changing the HSA doesn't modify an IOCDS.

I hope you can see that from a practical point of view, you must keep your HSA changes in sync with the IOCDS so that when the CPC is reset for any reason, it will come up with your latest changes rather than losing them. Changing the HSA and updating the IOCDS are two different tasks, each with their own system effects. It's important that those tasks are performed in lock-step with each other. All else leads to chaos and the fall of Civilization. A dynamic I/O configuration update is really just a way to implement a process that modifies both the IOCDS and HSA.

To change the I/O configuration you:

1. Create a new I/O configuration.
2. Modify the current I/O configuration to match the new one.
3. Write the validated I/O configuration to the IOCDS.

Just three things. Three! That's not so bad, is it? And while it isn't required mechanically (your gun, your foot), it should always be the objective to keep the current I/O configuration and the active IOCDS in sync.

Performing steps 2 and 3 in the order indicated might seem counter-intuitive, but it's the correct thing to do because step 2 also validates that the new I/O configuration is valid, thereby ensuring that a POR will load a good I/O configuration.

As I already mentioned, you can do this with CP commands, a pencil, and some paper. Cool beans, right? But let's face it. You have neither the time nor the desire to play Hide-and-Seek with the changes And then having found them, put them into the proper order. Oh, and recording the results so that you can reverse the changes, if necessary. Some enterprising individuals have written programs to help them do this. Of course, you're not in the application development business. Or, rather, your management doesn't want you to be in the application development business.

So given the importance of the I/O configuration to the continued proper operation of all of your LPARS, I recommend that you use HCD.

The HCD I/O Definition File (IODF)

Unlike IOCP, which can only write its results to an IOCDS, HCD/HCM creates file called the I/O Definition File, or IODF. The IODF contains all the information needed to create an IOCDS, alter the HSA, and to control the software's use of those devices.

Folks familiar with z/OS configuration will recognize the IODF. It has long been the way to define all the devices z/OS will use. z/VM, on the other hand, has been detecting (sensing) all of its devices without assistance since the early 1990s.

Some LPAR somewhere, whether on the local CPC or not, will hold the I/O configuration source in the form of a work IODF.

The HCD and HCM programs are used to update the work IODF. When those updates are complete, the IODF specific for the target CPC is exported from HCD and and sent to an LPAR on the target CPC.

Upon arrival, the work IODF is imported into the local HCD instance and converted to a production IODF. That production IODF is then activated through HCD commands. That causes a new IOCDS to be written and marked as active, and the current I/O configuration in the HSA is modified to match.

It's great; all of your I/O configurations for all of your machines in one place! Just keep in mind that you will switch to a different site in a disaster, so make sure each site has an LPAR with access to all of the site's I/O configurations.

When you activate the new I/O configuration, all of the affected LPARs are notified of they changes. Each will respond as it is configured to do so. By default, z/VM will bring online any new DASD it finds. Nothing will be automatically attached to the system or a user. Just varied online.

If you don't want that to happen, put this in SYSTEM CONFIG.

     Disable ,
        New_Devices_Initialized_When_Added

It won't take effect until the next IPL, so use the CP SET NEW_DEVICES command in the meantime. With this function disabled, you will have to manually VARY ON each desired device.

Prepare the Environment for Dynamic I/O

CPC authorizations

1. Enable dynamic I/O on the CPC:

You have to tell the CPC that you want to use dynamic I/O. This instructs the machine to load the token from the IOCDS (if there is one) at POR. Without the all-important token, you can't dynamically change the I/O configuration!

So check the box before you POR! Checking the box afterward has no immediate effect. You'll have to POR again.

I don't know why this is still necessary, decades after the function was introduced. I mean, you can still perform PORs to change the I/O configuration whether the token is loaded or not, and you still have to explicitly authorize an LPAR to be able to use the function! But I digress....

2. Select the LPAR that will be used to make changes to the I/O configuration and check the Input/Output (I/O) Configuration Control box, located on the Security tab of the LPAR's image profile.

To avoid incidents and accidents, ensure that only one LPAR on the CPC has this box checked!

3. Examine the IOCDS and HSA tokens as shown in the previous section. They should have tokens and those tokens should match. If not, write a new IOCDS with a token, and POR.

z/VM Configuration Changes

The ability of a virtual machine to manipulate or report on the I/O configuration of the CPC depends on its privilege class. Let's look at the array of functions and their associated IBM privilege classes:

• Read IOCDS (Class C or E)
• Write IOCDS (Class C)
• CP dynamic I/O commands (Class B)
• HCD dynamic I/O interface: DIAGNOSE 0x2AC (Class B)

Here's what's in the directory for the HCD/HCM server, CBDIODSP.

  IDENTITY  CBDIODSP  LBYONLY  256M  999M  BCEFG

• ☑ Class B is fine, as it represents all of the commands and functions related to real device management. I think mucking with the I/O configuration qualifies on that score!

• ☒ Class C is the most sensitive privilege class there is. I do not want the person doing I/O configuration to have privilege class C!

• ☒ Class E is redundant since class C gives access to the same dynamic I/O functions, and HCD doesn't need any of the other functions in class E.

• ☒ Class F isn't needed since HCD doesn't do any of things that need class F.

• ☑ Class G is good.

Because I don't want CBDIODSP to have class C, I will to turn to my faithful companion, SYSTEM CONFIG. In there I will be able to move the class C functions needed by HCD to a new privilege class of my choosing. For the purposes of this discussion, I choose class T.

1. Change the PRIV_CLASSES statement as shown, adding the needed functions to class T without removing them from their IBM-defined classes.

    PRIV_CLASSES        ,
         IOCP_Read CET  ,
         IOCP_Write CT

This change won't take effect until you re-IPL, so use the CP MODIFY PRIV_CLASSES command in the meantime.

2. Remove classes C, E, and F from CBDIODSP, replacing them with class T.

  IDENTITY  CBDIODSP  LBYONLY  256M  999M  BGT

I could have also moved DIAGNOSE 0x2AC to class T, but there's no point since I want to give all of the class B device QUERY commands to CBDIODSP. The important thing here was to avoid handing out privilege class C to a virtual machine that didn't need that much power.

3. CP needs to know if you intend to use HCD or not. In our case, we do, so SYSTEM CONFIG must be updated to have an extremely complicated IODF statement:

     IODF *

This format of the statement says that:

• HCD will be used to manage the CPC I/O configuration.
• The dynamic I/O-related CP commands will be disabled.
• The name of the I/O configuration definition file, IODFnn, will be obtained from the HSA.
• IODFnn PRODIODF will be on the the PARM disk.
• HCD will NOT control the z/VM I/O configuration. You will be able to VARY ON any available device.

If you find yourself in a bind and the system won't come up because you've cooked your goose, use the NOHCD IPL parameter on the SAPL screen to make CP come up as though IODF wasn't specified.

 STAND ALONE PROGRAM LOADER: z/VM VERSION 6 RELEASE 3.0
 
 DEVICE NUMBER:   5F5F       MINIDISK OFFSET:   00000000   EXTENT:  1
 
 MODULE NAME:     CPLOAD     LOAD ORIGIN:       1000
 
 --------------------------------IPL PARAMETERS--------------------------------
 NOHCD
 
 
 -----------------------------------COMMENTS-----------------------------------
 
 
 
 
 ------------------------------------------------------------------------------
 
 
 
9= FILELIST  10= LOAD  11= TOGGLE EXTENT/OFFSET

There are other ways to use HCD, but this is the simplest.

The information provided, and views expressed on this site are my own and do not represent the IBM Corporation.