Description of PUTFILES

Download count: 9 this month, 4648 altogether.
Downloads for PUTFILES:
  VMARC archive: v-17K

From Kris Buelens, IBM Belgium

PUTFILES is a REXX Pipelines stage that allows you to store records in CMS different files. Each record is prefaced by the target fileid.

For example, you get a big file in your reader that must be split into different CMS files. The target fileids are inside the reader file itself. Thanks to PUTFILES the task becomes easy and fast.
Example. Suppose :MEMBER cards separate the file data.

      'PIPE (end ?)',
         '|READER FILE' spid,             /* Get the RDR file */
         ... remove CCW, etc, maybe DEBLOCK NETDATA ...
         '|Fid: strFind /:MEMBER_/',      /* Find separator */
         '|SPEC W2 1.8 W3 10.8 /A/ 19',   /* Store fileid */
         '|JX: Juxtapose',                /* Join fileid with data */
         '|PUTFILES',                     /* Store the files */
         '?Fid:|JX:'                      /* Route data to Juxtapose*/

Changes in V2.2   (March 2021)


Berry van Sleeuwen encountered some problems with the data passed to various PUTFILES stages. To make debugging easier he added a NAME parameter so it becomes easier which PUTFILES instance encounters a problem.
I improved this further by adding a prefix to this name: a unique APn. or CLn. key. With this key you can easily find with ADDPIPE/CALLPIPE in PUTFILES REXX encounters the problem.
I also removed a the usage of TRACE REXX, a leftover from debugging time, only used when you tell PUTFILES to use a specific workunit for SFS.

Changes in V2.1   (Januari 2014)


We did not test the return code of DMSCOMM (the CSL routine to commit the work in SFS). As a result, servers that not return to CMS Ready continued with an open workunit, and updated files were left open, and these files remained locked, even for the server that left ot open. An example:

FPLDBK790E File locked by other user or other unit of work FPLMSG003I ... Issued from stage 2 of pipeline 2 FPLMSG001I ... Running ">>SFS 20131230 PTKGSU00 VMSYSU:MAINT.PTKGUI WORK" A reason to encounter this problem is "SFS filespace full".

Changes in V2.0   (September 2010)


The problems explained below with V1.2 have been fixed.

  • By default PUTFILES V2 works again like V1.1. That is it does not always use the >SFS and >>SFS stages if writing to SFS. PUTFILES calls the ">" or ">>" stages; but, when the filemode of file to write is written as a dirid, the > stage itself will call >SFS stage (similar for >>). Consequently, the default for >SFS is that many SFS connections will be used.
  • You can explictely request PUTFILES to use >SFS and >>SFS by using the USESFS parameter. PUTFILES will then either allow you to use workunits you obtained (and you commit/rollback them), either let PUTFILES use its own, single, workunit for all the files it will write to. In the latter case, PUTFILES will commit/rollback. Advantage with USESFS: only one SFS connection is used, you don't need to control the commit.

Summary: if your PUTFILES use may create many files in SFS, use its USESFS parameter to reduce the number of SFS connections.

For more information, refer to PUTFILES HELPPIPE.

Changes in V1.2   (September 2010)


When PUTFILES's target is SFS, it now explicitly uses >SFS and >>SFS instead of > and >>; this sometimes gives better messages when strange errors happen.

Furthermore, we detected that >SFS and >>SFS use by default a new workunit for each file created concurrently.  As PUTFILES can be used to create many files, it is easy to have for example 20 files open concurrently, consequently 20 APPC connections to the SFS server.
Many users and an SFS server with a relatively small USERS values in its DMSPARMS file may cause the SFS server (or end-user) to run out of connections.  The end-user may even end up in an ethernal wait (experienced with IPGATE).
Since V1.2, PUTFILES passes by default WORKUNIT DEFAULT to the >SFS or >>SFS stages, than only one connection to SFS is used.

Beware, I just found out that with WORKUNIT DEFAULT, you explicitely must commit the files written in SFS using PUTFILES.  I' working on a solution.  Till then,

  • either code PUTFILES WORKUNIT PRIVATE, and we work as before (that is with an APPC connection per created SFS file)
  • either change line 44 in PUTFILES REXX to become Workunit='PRIVATE' and the default of PUTFILES is again as before (that is with an APPC connection per created SFS file)
  • either commit all workunits after using PUTFILES:
      'PIPE ..... |PUTFILES |....'
      call CSL 'DMSCOMM retc reas'  /* Commit all the work*/
  • either get your own workunit and commit it at the end:
      call csl 'DMSGETWU retc reas WID' /* Get a new workunit */
      'PIPE ..... |PUTFILES WORKUNIT' wid '|....'
      call CSL 'DMSCOMM retc reas WID'  /* Commit the work from PUTFILES*/
      call CSL 'DMSRETWU retc reas WID' /* Tell CMS we're done with PUTFILES*/

For more information, refer to PUTFILES HELPPIPE.