Moving CMS Files with FTP
Every now and then somebody tries to FTP a CMS file from a customer system to an IBM system. The usual scenario is that someone FTPd the CMS file from the customer CMS system to a laptop, then bounced it through one or more intermediate systems, and then FTPd it up to the IBM CMS system. Bytes move, but the file that lands on the target CMS is unusable. Shortly after that I receive an e-mail asking what went wrong and how to fix it. Thus, this article.
Any attempt to move a CMS file from one CMS system to another needs to recognize this basic truth: the CMS file system is a record-oriented file system. On CMS, a file is a sequence of records. Thus every CMS file consists of bytes of data, a record format, and record boundaries. To move the file successfully from one CMS to another, one must move all three of those. Moving just the bytes is not good enough.
The intermediate platforms in the opening scenario rarely have record-oriented file systems. Windows, for example, preserves only the bytes. It's the same with Linux. To preserve the record format and the record boundaries, one must pack or package the CMS file first, using CMS commands issued on the source CMS system. The output of the packing or packaging exercise will be a CMS file we'll call the movable file. The packing or packaging operation puts all three elements of your payload file -- its bytes, its record format, and its record boundaries -- inside the movable file in a way they can be reconstructed by an unpack or unpackage operation one does on the target CMS.
The movable file itself can be FTP'd through the intermediate systems in binary without penalty. Its record format and record boundaries will be lost, but they are so utterly simple that they can easily be reconstructed by hand on the target CMS. Once the movable file is reconstructed, we unpack it so as to extract the payload file.
In case all of this sounds familiar, what I've described above is a lot like creating a .zip archive, FTPing the .zip archive through the network, and then unzipping the .zip archive on the other side. However, please do not send us CMS files packaged into a .zip archive. Believe it or not, sometimes we can't un-ZIP them. Even if we could, owing to the considerations described above, it's not clear the files would survive.
Here are two different means for packing or packaging one or more CMS files in such a way that we can FTP them through a string of intermediate systems with safety.
Using COPYFILE with the PACK option
If you have just one CMS payload file to move, the CMS COPYFILE command with the PACK option is a great way to make your payload ready for transmission. COPYFILE ( PACK both applies data-compression to your payload's bytes and creates an output file you can move through the network with impunity, as long as you always move it in BINARY transfer mode.
So, supposing the CMS file you need to move is called MY FILE A, here's what you would do:
- COPYFILE MY FILE A MYFILE PACK A ( PACK REPLACE
- FTP the file MYFILE PACK A as much as you want, among as many different systems and system types as you want, as long as you always move it in BINARY transfer mode
- Eventually your last FTP operation will result in your placing the file MYFILE PACK onto some CMS minidisk, let's assume it's the A-disk,
- PIPE < MYFILE PACK A | fblock 1024 00 | > MYFILE PACK A F 1024
- COPYFILE MYFILE PACK A MY FILE A ( UNPACK OLDDATE REPLACE
Remember I told you it's easy to reconstruct the record format and the record boundaries of the movable file? That's what the PIPE command above does. Don't forget that crucial step, otherwise COPYFILE ( UNPACK won't work.
If you do the above steps exactly as I've illustrated, your CMS payload MY FILE A will survive just fine.
Using VMARC
A publicly available CMS utility program called VMARC is kind of like the "zip" program of the CMS world. Using VMARC we can cram one or more CMS files into a single CMS file called a VMARC archive file. Just like a COPYFILE ( PACK output file, a VMARC archive file has the property that a cascade of FTP operations can move it safely through the network, provided it's always moved in BINARY transfer mode. Also, like COPYFILE ( PACK, the information inside the VMARC archive file is data-compressed. If you don't have the VMARC tool, you can get it from my VMARC page.
Syntax
The syntax for your VMARC command(s) would be this:
VMARC PACK infn inft infm outfn outft outfm ( options
where:
- infn inft infm is your payload file, e.g, D052107 T083242 A
- outfn outft outfm is the VMARC archive file you are building, e.g., MYDATA VMARC A
- options are the options for the command:
- Use ( REPLACE for the first file. This creates the archive.
- Use ( APPEND for the rest of the files. This appends to the archive.
Example
Here's an example of how to put three MONWRITE files into one VMARC archive file. Notice for the first command, we use REPLACE to create a new archive. For the next two commands, we use APPEND to append to the archive we created.
The result of this sequence of commands is file MYDATA VMARC A, which contains the three MONWRITE files, data-compressed.
Moving the VMARC Archive File
Once you create your VMARC archive file MYDATA VMARC A, move it through the network like this:
- FTP the file MYDATA VMARC A as much as you want, among as many different systems and system types as you want, as long as you always move it in BINARY transfer mode
- Eventually your last FTP operation will result in your placing the file MYDATA VMARC onto some CMS minidisk, let's assume it's the A-disk,
- PIPE < MYDATA VMARC A | fblock 80 00 | > MYDATA VMARC A F 80
- VMARC UNPK MYDATA VMARC A = = A ( OLDDATE REPLACE
Remember I told you it's easy to reconstruct the record format and the record boundaries of the movable file? That's what the PIPE command above does. Don't forget that step, otherwise VMARC UNPK won't work.
If you do the above steps exactly as I've illustrated, everything you put into your VMARC archive file will pop out on the other side when you issue VMARC UNPK.