VM IMAP Beta Release 1.6.1 Installation

IMPORTANT NOTE: The index structure used by Beta Release 1.6 and higher is not compatible with the index structure used by earlier beta levels. If you were running a level of the IMAP server prior to Beta Release 1.5, you MUST delete and re-enroll all users so that they will end up with the new index structure. If you're running level 1.5 you can either delete and re-enroll all users or you can use the IMAP5TO6 sample exec to convert their existing indexes in the mailstore to the new format. Don't forget - if you choose to change index formats by deleting and re-enrolling users, deleting a user causes all their existing mail and folders to be removed from the mailstore.
    - minimum system requirements
       o  VM 2.3.0, CMS14, TCP/IP 310
       o  LE 1.8
       o  SMTP APAR PQ31576 required
       o  SMTP APAR PQ41591 recommended
       o  CMS APAR VM61477 recommended
       o  CMS APAR VM62318 recommended
       o  CMS APAR VM61821 and TCPIP APAR PQ17871
          only required if you want to use a translate table
          other than the default one (note - you should also
          review PQ26583 and PQ31152)
       o  CMS APAR VM62679 recommended
       o  CP APAR VM62662 recommended
    - requires an "IUCV *SPL" directory statement to allow the server
      to connect to the *SPL system service in order to asynchronously
      read in mail as it arrives in the server's reader
    - requires an "IUCV ANY" (or an "IUCV sfs_server_id") directory
      statement to allow it to connect to the Mailstore SFS server
      (the IMAP server also must have SFS admin authority to the
      SFS filepool)
       Note:  This directory statement for the IMAP server will
              allow it to connect to the SFS server as opposed to
              the usual method of having an IUCV ALLOW statement in
              the SFS server's directory.  This is necessary because
              we do not want IMAP users able to directly connect to
              the SFS server and corrupt the directory structure or
              the data residing in the filepool.  If you have other
              users that will need to access the SFS server (eg:
              backup machines) you will need to add an
              IUCV sfs_server_id to their directory entries too.
    - requires an "IUCV ALLOW" to enable the IMAP server for incoming
      APPC/VM conversations which will be initiated by any IMAP
      administrators that have been specified in the CMS private
      resource registration file on the IMAP server's A disk and which
      use the queue-based administrative interface that is provided
      in the IMAP SAMPEXEC
    - requires an "OPTION MAXCONN 2000" directory statement
      to allow up to 2000 IUCV connections to the SFS server.
    - if running with AUTHENTICATE CP, requires privilege class B
      in order to issue DIAG 84 to validate passwords.  If running
      with AUTHENTICATE ESM, this may not be required, but some
      authorizations will need to be configured on the ESM.
    - sample IMAP server directory entry might look like:
         ACCOUNT XXX
         IUCV ALLOW
         IUCV ANY
         IUCV *SPL
         MACHINE XC
         CONSOLE 0009 3215 T
         SPOOL 000C 2540 READER *
         SPOOL 000D 2540 PUNCH A
         SPOOL 000E 1403 A
         LINK MAINT 0190 0190 RR
         LINK MAINT 019E 019E RR
         LINK TCPMAINT 0592 0592 RR
         MDISK 0191 3390 3316 2 K1E41A MR
    - requires a $SERVER$ NAMES file (private resource registration
      file) on an accessed disk with the following VMIPC entry to
      list any virtual machines that will be using the queue-based
      administrative interface (IMAP SAMPEXEC) to issue commands to
      the IMAP server (such as ENROLL USERID):
                 |  :nick.VMIPC                    |
                 |    :list.user1 user2 user3 ...  |
    - requires an IMAP CONFIG file (userid CONFIG is also
      allowed) on an accessed disk which specifies
      configuration information for the IMAP server (FILEPOOLID and
      BADFILEID are required IMAP configuration file statements to
      specify the filepool id of the mailstore and the userid to which
      bad mail files should be sent, if these are not configured
      the server will not start).
      The following are the currently supported statements that can
      be specified in the IMAP CONFIG file:
       FILEPOOLID userid        - userid of the SFS server containing
                                  IMAP's mailstore.  No default.
       BADFILEID userid         - userid to whom reader files are
                                  transferred when the IMAP server has
                                  difficulty delivering them.  No
       MAILORIGINID userid ...  - userids from whom mail arriving in the
                                  reader must originate.  Default is
                                  obtained from the TCPIP DATA file.
                                  If not present in the TCPIP DATA
                                  file, defaults to SMTP.  Up to 32
                                  userids may be listed.
       TRACE ALL                - turns all tracing on (CODEFLOW,
                                  SOCKLIBCALLS, and SOCKETIO).
       TRACE CODEFLOW           - turns module entry/exit tracing
       TRACE SOCKLIBCALLS       - turns socket library call tracing
       TRACE SOCKETIO [ipaddr]  - turns tracing of all data sent
                                  or received on a socket on.
                                  If an IP address is given, only
                                  sockets active from that IP address
                                  will be displayed.
       AUTHENTICATE CP|ESM      - whether userids should be
                                  authenticated by CP (the directory)
                                  or by an external security manager.
                                  Default is CP.
       PORT integer             - TCP port number on which the IMAP
                                  server will listen.  If omitted IMAP
                                  listens on port 143.
       READERTHREADS integer    - number of threads to create to
                                  handle incoming reader files.
                                  Default is 1 thread.
       IDLETIMEOUT integer integer - first integer is the amount of
                                     time (in seconds) that an
                                     unauthenticated connection can
                                     be idle (have no activity).  The
                                     second integer is the amount of
                                     time (in seconds) that an
                                     authenticated connection can be
                                     be idle.  Default values are
                                     60 and 0 (0 means infinity).
                                     RFC 2060 section 5.4 restricts
                                     the second number to a minimum
                                     of 1800 (30 minutes).
       ENROLLDEFAULTS integer integer - first integer is the number of
                                         blocks to allocate for a
                                         user if no value is provided
                                         on the ENROLL command.  The
                                         second number is the storage
                                         group the user should be
                                         enrolled in.  Defaults are
                                         100 and 2 respectively.
      Blank lines are allowed, and comment lines must begin with
      a semicolon.
    - the PROFILE EXEC for the IMAP server should be updated to activate
      the C runtime library (GLOBAL LOADLIB SCEERUN), and (optionally)
      to start up the IMAP server (IMAPMAIN MODULE)
    - to provide a custom translate table follow the directions in
      the TCPIP documentation for creating a TCPXLBIN file (or
      copy an existing file with the desired translation).  The
      IMAP server follows the standard hierarchy for translation
      tables.  It first looks for IMAP TCPXLBIN, if that is not
      found it looks for STANDARD TCPXLBIN.  Lastly if neither
      are found it uses a hardcoded translate table that is
      adequate for most purposes.  It is strongly recommended
      that IMAP be configured to use the same translation table
      as SMTP is using.
    - we strongly recommend a dedicated SFS server be used as
      the IMAP mailstore.  For details on setup and configuring
      an SFS filepool please refer to the Filepool Planning,
      Administration and Operation manual for your VM release.
      What we're documenting here are the differences from
      a "normal" filepool setup.
    - requires an "IUCV *IDENT" directory statement to identify
      itself as a resource owner, but no "IUCV ALLOW" since we do
      not want enrolled user's to be able to connect to the server
      and corrupt the directory structure or the data residing in
      the filepool
    - requires an "OPTION MAXCONN 2000 APPLMON" directory statement
      to allow up to 2000 APPC connections to the server.  One
      APPC connection is used for each active user.  2000 is a
      reasonable initial default.  If you expect to have more than
      2000 concurrent users you should adjust it upward to match
      the expected number of users.  This number should be the
      same for the IMAP server and the IMAP mailstore.
    - make sure that the IMAP server virtual machine has SFS admin
      authority for the filepool
    - sample IMAP mailstore SFS server directory entry might
      look like:
         ACCOUNT G28G GXX/BOX 002
         MACHINE XC
         CONSOLE 0009 3215 T
         SPOOL 000C 2540 READER *
         SPOOL 000D 2540 PUNCH A
         SPOOL 000E 1403 A
         LINK MAINT 0190 0190 RR
         LINK MAINT 019D 019D RR
         LINK MAINT 019E 019E RR
         MDISK 0191 3390 3336 1 K1E419 MR
         MDISK 0301 3390 2906 ? K1E400 MR
         MDISK 0302 3390 3336 ? K1E400 MR
         MDISK 0303 3390 3337 ? K1E400 MR
         MDISK 0304 3390 3338 ? K1E400 MR
         MDISK 0305 3390 3336 ? K1E418 MR
    - an administrative virtual machine will be used to enroll and
      delete users in the IMAP Mailstore as well as issue other
      administrator commands (eg: shutdown the server, set an alert,
      etc.).  This may be the systems programmer's personal id or
      someone else's id or an id purposely created for IMAP
    - make sure that the administrator userid(s) are specified in the
      private resource registration file ($SERVER$ NAMES) on the
      IMAP server virtual machine.  Keep in mind that each userid
      listed in $SERVER$ NAMES has access to the full suite of
      administrator commands and can DELETE users, shut the server
      down, or in other ways cause havoc.
    - requires the IMAP SAMPEXEC to communicate with the IMAP server
      using the queue-based administrative interface
    - IMAP SAMPEXEC should be renamed to have a filename matching the
      IMAP server virtual machine userid, a filetype of exec and
      then must reside on an accessed disk so that the administer can
      run the exec to issue administrative commands
       Example:  For an IMAP server with a userid of IMAP, the exec
                 would be renamed to IMAP EXEC and "IMAP ENROLL SMITH"
                 would enroll userid SMITH into the IMAP mailstore.
    - the following are the currently implemented administrator
       ALERT [optional text]
        If no text is supplied then the current alert is cleared.
        If text is supplied it becomes the alert text that will
        be forwarded to each client on their next transaction.
       CP command
        The specified command is passed UNCHANGED (it is
        not uppercased for example) to CP for execution via
        a DIAGnose 08.  The command's return code is returned.
        This is useful for spooling console closed, etc..
       DELETE userid
        Un-enroll the specified userid
       ENROLL userid [blocks [storage_group]]
        Enroll a userid in the mailstore, optionally an initial
        block allocation and storage group can be specified.
        If omitted, their values default to those specified
        on EnrollDefaults in the config file.
        Returns a list of supported commands
       MODIFY userid +/-blocks
        Increase or decrease the number of blocks a user is
        authorised to consume in the mailstore.
       SHAREFOLDER to_user foldername owning_user foldername
        This adds an entry in the to user's mailbox index table
        for a folder with foldername.  The entry actually points
        to owning_user's folder (see usage note below).
        Shut down the IMAP server
        Start an additional thread to process reader files.
       STATS [RESET]
        Return a list of server statistic counters, and optionally
        reset the counters.
        TRACE ALL [ON] turns on all (codeflow, socklibcalls and
        socketio (for all IP addresses)) tracing.  TRACE ALL OFF
        turns off all (codeflow, socklibcalls and socketio)
        tracing.  TRACE CODEFLOW [ON] turns on just module entry and
        exit tracing.  TRACE CODEFLOW OFF turns off module entry and
        exit tracing.  TRACE SOCKETIO [ON] turns on display of all
        data sent and received on a socket.  You can optionally
        specify an IP address.  Only sockets active from that IP
        address will have their socket data displayed.  TRACE
        SOCKETIO OFF turns off socket I/O tracing.  TRACE
        SOCKLIBCALLS [ON] traces each socket library call.  TRACE
        SOCKLIBCALLS OFF turns off socket library call tracing.
    - the needed SFS authorities must be in place before issuing
      the SHAREFOLDER command.  The folder owner must have
      granted read access to their MAILBOX INDEX file in their
      highest level directory and also read access to each file
      in the directory being shared.  SHAREFOLDER does not attempt
      to issue the needed grant commands because there are a
      number of SFS ESMs each of which handles the granting of
      authorities in their own way.  If you would find an exit
      useful at this point (so the granting may be automated)
      please let us know.
    In general we recommend a dedicated SMTP server for use with
    IMAP.  This does not require a dedicated stack, since you can
    use multihoming support.  However you must be at FL 320 before
    the multihoming support works with SMTP.  We do have beta
    customers running on FL 310, using the command exit on their
    primary SMTP (with FL 310 and FL 320) or not using our IMAPCMDX
    at all since they run with a mailer (they have developed similar
    function for use with their mailer (IMAP determines who the mail
    is destined for by examining netdata text unit INMTUID (x1002),
    the important thing is that it be set correctly)).  So the
    recommendation to use a dedicated server is flexible.
    - if you are not calling your IMAP server IMAP you must edit
      IMAPCMDX and update the servername variable (while in there,
      verify the at sign character is defined as expected for
      your code page (generally only a concern in Scandinavia))
    - place the IMAPCMDX EXEC and IMAP123 REXX files on SMTP's
      191 (or somewhere in it's access order, eg: TCPMAINT's 198)
    - in order to have mail sent to the IMAP mailstore, the SMTP
      virtual machine must be configured to send local mail to
      the IMAP virtual machine.  This is done using the command
      exit for the PUNCH command.  Add the following to your
      SMTP's config:
    - this will cause SMTP to call our exit which will send a
      file over to the IMAP virtual machine whenever local
      mail arrives
    - the IMAP virtual machine will determine if the mail is for a
      registered IMAP user and automatically place it in the INBOX
      for that user (if it is not for a registered IMAP user, then
      it will transfer it directly to the user)
    - once all of the appropriate setup work has taken place, start
      the IMAP server by invoking the IMAPMAIN MODULE from the
      command line or profile of the IMAP server virtual machine
    - IMAPMAIN performs the following:
       -- read and process the configuration information from the
          configuration file (it first looks for a serverid CONFIG *
          file, if that is not found it looks for an IMAP CONFIG *
       -- create a thread to process console input
       -- create a thread to process mail arriving through the spool
       -- create a thread to handle administration requests
       -- create a thread to listen for new connections on the IMAP
          port (this thread will create an additional thread for
          every new client connection to the server)
    - when the IMAP server is up and running, the administrative
      interface is used to enroll new users
    - from an administrative virtual machine, invoke the sample exec
      (which has been renamed from IMAP SAMPEXEC to the name of the
      IMAP server with filetype exec) to enroll a user
         eg: IMAP ENROLL SMITH
    - this will cause the IMAP server to perform the following:
       -- enroll the userid in the SFS filepool
       -- create an INBOX directory for the user
       -- create the MAILBOX INDEX file in the user's root directory
          to contain a list of the user's folders and add the INBOX
          folder to the table
    - note that the CSL routines called to Enroll and Delete users
      can only be called synchronously.  Bulk enrolls and deletes
      should not be done at a peak time for client use since their
      response will be impacted.
    - there is only a single thread processing administrator requests.
      Multiple concurrent requests will be processed synchronously.
    - if the SMTP machine has been configured to route mail to the
      IMAP server then all local mail for the system will be routed
      over to the IMAP server.
    - the IMAP server's mail arrival thread will use the *SPL system
      service to read in the note data and, if the mail is destined
      for an enrolled IMAP user, it will add the message to the user's
      INBOX by doing the following:
       -- generating a UID (unique identifier) message attribute for
          the message
       -- parsing the note and writing the data into a file in the
          user's INBOX directory using the UID for a filename and
          a filetype of NOTE (if the note was a multipart note an
          index of the parts will also be created also using the UID
          for a filename and a filetype of PARTINDX)
       -- adding an entry to INBOX's NOTE INDEX file for the new
       -- updating the entry for INBOX in MAILBOX INDEX to have
          updated counts of total messages, unseen messages, etc.
    - if the note is for a user that is not a registered IMAP user
      the file is transferred to their virtual reader.
     - while the IMAP server is up and running, users can use any
       client that supports the IMAP4rev1 protocol (RFC2060) to
       connect to the IMAP server and perform IMAP client functions
       (such as creating/deleting/renaming mailboxes, checking for
       new messages, deleting messages, etc.)
     - in order for the client to get into the authenticated state,
       it is necessary to send a LOGIN command (rather than the
       AUTHENTICATE command).  The initial release of the server
       requires that the userid and password be a valid VM
       userid/password pair and the userid must be enrolled in
       the IMAP mailstore (LOGIN processing checks that the user
       has an INBOX directory in the filepool)
     - The IMAP server supports all of the IMAP commands documented
       in RFC 2060 with the following exceptions:
        -- optional AUTHENTICATE command is not supported (the LOGIN
           command must be used to get to the authenticated state)
           (we hope to provide an AUTHENTICATE exit in the future
           that would allow a site to use what ever form of
           authentication they wish).
        -- there are no experimental commands supported
    - we recommend that the IMAP server initially be given a storage
      size between 128M and 256M.  If you're expecting heavy usage
      (>3000 concurrent users) you should increase this amount.
      Beta level 1.6 requires significantly less storage to support
      a given workload than earlier levels did (the threading logic
      has been redesigned, each client connection no longer requires
      a dedicated thread).