IBM: Description of PIPEDDR

Description of PIPEDDR

Download count: 8 this month, 8091 altogether.
Downloads for PIPEDDR :
VMARC archive: v-102K
zip archive: z-109K

PIPEDDR EXEC

The PIPEDDR EXEC can dump a disk into a regular CMS file, to an ftp server, to a file in an NFS share (which is mounted to BFS, the CMS Byte File System), or via TCP/IP directly to a disk on a remote system. Restore a disk from a PIPEDDR output file via one of the same methods. The data is packed or can also be compressed further, saving network bandwidth or disk space. The data can also be encrypted either on disk or as it is sent over the network. A remote disk can be kept in sync with a local disk using the remote update function.

The PIPEDDR EXEC can save and restore disk images using either the raw disk I/O support available in CMS Pipelines or the Pipelines interface to the CMS DDR command. The internal structure of each kind of disk image are different and are not compatible with each other. The Pipelines raw disk I/O support uses the Pipe stages TRACKREAD and TRACKWRITE (for ECKD disks) or FBAREAD and FBAWRITE (for FB-512 disks.) These stages are part of either z/VM® 6.4 and later or the optional "Runtime Distribution" of CMS Pipelines. The interface to the CMS DDR command uses the DDR Pipelines stage.

The exec dumps an entire disk into a regular CMS file. The operating system format of the disk or the type of data on it does not matter. The disk is read in binary track by track or block by block. The output data by default is written in packed format compatible with the PACK option of the CMS COPYFILE command and is in fixed record format with 1024 byte records. This allows easier interchange of the backup files with non-mainframe systems. These files can be used for backups of disks or restored on another system. There are also options to compress the output to make the file even smaller. See the notes section of the help file for more information on compressing the output.

The output file created by this exec has meta data about the source disk as the first record. The contents of the disk follow in the rest of the records. This ensures that the disk is restored to a device of the same type and size. Data must be restored to the same type of device using the same disk data encoding method. In other words, ECKD devices (3390) must be restored to ECKD devices and FB-512 devices must be restored to FB-512 devices. It is not possible to convert the data on one device type to another device type using this exec.

The disk can also be transferred over a TCP/IP connection directly to a receiving PIPEDDR exec listening to an IP port on a remote system. The default is to send the entire disk. If the disk was transferred previously, a PIPEDDR option will send just the changes to the blocks or tracks to the remote system instead of the entire disk. Recent updates to CMS Pipelines and TCPIP allow optional secure (TLS) remote sessions. It is recommended that the same version of the PIPEDDR exec be used on both ends of the TCP/IP connection. PIPEDDR also supports sending the output file directly to an ftp or ftps (secure ftp) server. The file created on the ftp server is in the same format as the CMS file. A disk can be restored from a file hosted by either an ftp server or an HTTP (web) server. Secure connections are supported.

By default, PIPEDDR will display on the console timestamps and messages about the progress of the disk operation as it processes a disk. The default behavior is to display a message for every 10% of disk that has been processed.

If the disk contains sensitive data, the disk contents can be encrypted using the CPACF hardware encryption processor. You must obtain the KM Package from the VM download library to enable encryption. The KMRTNS CSLLIB is used from the VMARC file. These CSL routines require your machine to have the CPACF feature enabled to use the built in hardware encryption and decryption.

Dumping or restoring to FTP supports 3 different FTP pipeline stages. The newest ftp stage is built in to Pipelines and is available if secure TCP/IP connections are supported by CMS Pipelines. (APAR VM66365 for z/VM 7.1, included in z/VM 7.2 and later.) This stage supports both secure and unsecured ftp sessions. The other ftp stages supported by PIPEDDR are not built-in to Pipelines; they are loaded from a separate module or file.

The exec also supports two other forms of compaction, TERSE and ZLIB. The TERSE option can only be used if the TERSE Pipelines stage is available. This allows the exec to create output files that are smaller or send less data over the network. The TERSE Pipelines stage is part of the PIPSYSF filter package which is now available from the Marist Pipelines page. See the PIPSYSF Filter Package page for more information and a link to download the module.

The ZLIB compression option requires the zlib Pipeline stage which is found in in the ZLIBSTG MODULE file. You can download the file "fplgcc.vma" which is a VMARC format file downloadable from John Hartmann's space on GitHub.

If the CMS DDR format is used, the DDR module supports 2 compression options, COMPRESS and LZCOMPACT, which may provide an acceptable level of compression.

There is also a copy option which preforms a disk to disk copy, with automatic linking of the source and target disks. If the flashcopy option is specified, the exec attempts to copy the disks using the CP FLASHCOPY command. If that command fails or your user id is not allowed to use this command, the exec falls back to a disk to disk copy. The copy option is no different than copying a disk using DDR, and in fact just the DDR module is used if the CMSDDR option is specified on the copy command.

A help file is included in the package and it contains a detailed description of all of the options. Enter HELP PIPEDDR on the CMS command line for usage information. Be sure to also download the zip file which contains a pdf document. This document includes all of this description, all of the help file contents, plus additional details.

Feedback: Bruce Hayden IBM (retired)

Some examples of how to use PIPEDDR.

To dump a minidisk to a file with the default name:

PIPEDDR DUMP MAINT 19E

This creates a file named MAINT DISK019E A

To dump a minidisk to a file and terse the output:

PIPEDDR DUMP MAINT 19E (TERSE

Another way is to set the default compression format to TERSE and then dump the disk:

PIPEDDR SET COMPACT TERSE
PIPEDDR DUMP MAINT 19E

To dump a minidisk to a file including the CRC:

PIPEDDR DUMP MAINT 19E (CRC

To dump the same disk to a different filemode:

PIPEDDR DUMP MAINT 19E TO FILE = = E

To dump a minidisk to a file in CMS DDR format:

PIPEDDR DUMP MAINT 19E (CMSDDR

To restore the file to the same disk:

PIPEDDR RESTORE MAINT 19E

To restore the file to a different disk and skip the prompt:

PIPEDDR RESTORE CMSUSER 191 FROM FILE MAINT DISK019E A (NOPROMPT

To dump a minidisk to a file and encrypt the data:

PIPEDDR DUMP MAINT 19E (CIPHER AES KEY(Super Secret KEY)

To restore it:

PIPEDDR RESTORE MAINT 19E (CIPHER AES KEY(Super Secret KEY)

To send an entire minidisk over the network. On the receiving node, enter:

PIPEDDR RESTORE MAINT 19E FROM REMOTE

This displays the port number where it is listening for the remote data.

To force it to listen to TCP/IP port 12345:

PIPEDDR RESTORE MAINT 19E FROM REMOTE (PORT 12345

On the sending system use (where pppp is the listening port):

PIPEDDR DUMP MAINT 19E TO REMOTE nodeid.example.com:pppp

To just update a disk that was copied previouslly:

PIPEDDR DUMP MAINT 19E TO UPDATE nodeid.example.com:pppp

The connection should be made and the disk sent over.

To verify that the label of the disk image from the remote system is correct, on the receiving side enter:

PIPEDDR RESTORE MAINT 19E FROM REMOTE (VERIFYLABEL MNT19E

To send a minidisk to a file named maint.disk019e on an ftp server:

PIPEDDR DUMP MAINT 19E TO ftp://hayden:password@server.example.com

To restore a minidisk from file disk.dump on an ftp server:

PIPEDDR RESTORE MAINT 19E FROM ftp://hayden:password@server.example.com/disks/disk.dump

To do the same thing but from an http server:

PIPEDDR RESTORE MAINT 19E FROM http://server.example.com/disks/disk.dump

To dump a minidisk to a file named maint.disk019e on an NFS server, first the nfs directory must be mounted to your virtual machine using OPENVM MOUNT. Assuming it is mounted at /home/maint/mnt, the command is:

PIPEDDR DUMP MAINT 19E TO BFSPATH /home/maint/mnt

Change log of recent changes, latest changes first

Version    Change Description
-------    ------------------
V1.8.2     Add VERIFYLABEL option.
V1.8.1     Add error messages for some missing keywords in options.
V1.7.16    Fix error showing progress messages with older Pipelines.
V1.7.15    Support a CMS KEYVAULT for ftp id and password.
V1.7.14    Show message for unsecured remote connection.
*          Add timestamp to dump/restore progress messages.
*          NOIMSG and SLEEP options, improve HTTP client code.
V1.7.13    Add some options to help testing.
V1.7.12    Handle encoded urls, encode password for ftp stage.
V1.7.11    Add FTPSITE option to send FTP site command
V1.7.10    Allow delimiters of ' or " for cipher key string, fix 3380.