SSL Server Becomes Unresponsive During Operation
APAR Identifier ...... PM59015 Last Changed ........ 13/04/25 SSL SERVER BECOMES UNRESPONSIVE DURING OPERATION Symptom ...... IN INCORROUT Status ........... CLOSED PER Severity ................... 2 Date Closed ......... 12/09/28 Component .......... 5735FAL00 Duplicate of ........ Reported Release ......... 540 Fixed Release ............ 999 Component Name TCP/IP V2 FOR V Special Notice Current Target Date .. Flags SCP ................... Platform ............ Status Detail: SHIPMENT - Packaged solution is available for shipment. PE PTF List: PTF List: Release 540 : UK82210 available 12/10/03 (1202 ) Release 610 : UK82211 available 12/10/03 (1301 ) Release 620 : UK82212 available 12/10/03 (1301 ) Parent APAR: Child APAR list: ERROR DESCRIPTION: After handling a varied number of connections, the SSL server can become unresponsive to connection or administrative command requests. Initial investigation of this problem indicates this condition likely becomes manifest due to underlying socket error handling or timing problems, and likely is not related to a given connection load. LOCAL FIX: None. PROBLEM SUMMARY: **************************************************************** * USERS AFFECTED: Users running high-traffic SSL enabled * * services. * **************************************************************** * PROBLEM DESCRIPTION: * **************************************************************** * RECOMMENDATION: APPLY PTF * **************************************************************** Due to a CMS sockets implementation error, when socket write() or send() operations could not be executed by the TCP/IP stack, incorrect return code and 'errno' values were passed to the SSL server. In particular, this condition occurred when the DATABUFFERPOOLSIZE buffers in the TCPIP stack were exhausted. Because of the incorrect socket error notifications encountered by the SSL server, its ensuing actions, based on these, can result in hung sessions, dropped sessions, or a server abend. PROBLEM CONCLUSION: Erroneous CMS sockets I/O operation return codes have been corrected by dependency APAR VM65148. This APAR corrects a condition in the SSL server that could result in a server abend if received encrypted data is not correct. In addition, support for SSL server coordinated dump support is introduced. This support provides a mechanism for acquiring virtual machine dumps for the SSL server (and, its associated TCP/IP stack and DCSS agent servers) when selected error conditions are encountered by an SSL server. Because a high degree of interdependence exists among this set of virtual machines, the acquisition of a set of coordinated virtual machine dumps for these servers can provide more useful information for diagnostic purposes. SSL server coordinated dump support is controlled by the VMSSL command VMDUMP operand, also introduced with this APAR. ================================================================ The information that follows will be included in any future updates to the following publication(s): SC24-6238-02 -- z/VM: TCP/IP Level 620 Planning and Customization SC24-6238-01 -- z/VM: TCP/IP Level 610 Planning and Customization SC24-6125-05 -- z/VM: TCP/IP Level 540 Planning and Customization --------------------------------------------------------------- Chapter 18. Configuring the SSL Server (Level 620) Chapter 20. Configuring the SSL Server (Levels 610, 540) Section: VMSSL Command (All levels) The VMSSL command syntax diagram is updated to include the VMDUMP operand: .--Error---. >>--VMDUMP---+----------+--------------------------->> |--Error---| '--Socket--' The VMSSL command 'Operands' section is updated to include documentation for the VMDUMP operand: Operands VMDUMP error_type instructs the SSL server to create a virtual machine dump when an error of the indicated type is encountered. In addition, the affected server initiates the creation of dumps for its associated TCP/IP stack and DCSS agent servers, when conditions allow for this. error_type identifies the type of errors for which a virtual machine dump is to be created. Possible values for 'error_type' are: Error specifies that a dump is to be created for an q unexpected, severe error condition. This is the default. Socket specifies that a dump is to be created for unexpected socket-related errors only. Notes: * The SSL server (or server pool) requires authorization use the non-general version of the CP FOR command. IBM-defined privilege class C provides this authorization. * The virtual machine dumps created by using the VMDUMP operand are processed using SYSTEM operand of the CP VMDUMP command (thus, dumps are transferred to the user specified on the SYSTEM_USERIDS CP configuration statement of the SYSTEM CONFIG file). --------------------------------------------------------------- Chapter 1. Planning Considerations Section: User ID Privilege Class Considerations Table 1. TCP/IP Server and User ID Assigned Privilege Classes is updated to include: Privilege User ID Class Pertinent Commands and Capabilities ------------------------------------------------------------- SSLnnnnn, C CP FOR command capability, to SSLSERV accommodate coordinated virtual machine dump processing ================================================================ Documentation for the following new and modified messages will be included (as noted) in any future updates to the following publications: GC24-6237-02 -- z/VM: TCP/IP Level 620 Messages and Codes Chapter 18. SSL Messages Chapter 20. TCP/IP Utilities GC24-6237-01 -- z/VM: TCP/IP Level 610 Messages and Codes Chapter 18. SSL Messages Chapter 20. TCP/IP Utilities GC24-6124-04 -- z/VM: TCP/IP Level 540 Messages and Codes Chapter 18. SSL Messages Chapter 20. TCP/IP Utilities --------------------------------------------------------------- Chapter 18. SSL Messages Section: SSL Server Messages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message DTCSSL0214W is added for Level(s): 620 610 540 DTCSSL0214W Initiating dump of server: <userid> Explanation: The SSL server, configured with the VMDUMP operand, has encountered an error condition. The VMDUMP command is being invoked on the indicated server (<userid>) to collect diagnostic information about this error. The affected server can be an associated TCP/IP stack or DCSS agent server, or the SSL server itself. Based on conditions when the error was encountered, virtual machine dumps for any of these virtual machines might be produced. System Action: After VMDUMP processing has completed for the affected SSL server, the server terminates. Other servers continue processing upon completion of the VMDUMP command. System Programmer Response: Collect and process the generated dumps, save any existing problem information, and contact the IBM support center for assistance. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message DTCSSL0215W is added for Level(s): 620 610 540 DTCSSL0215W Bypassing dump processing for server: <userid> Explanation: The SSL server, configured with the VMDUMP operand, has encountered an error condition. Because VMDUMP processing already has been performed (since the reporting server was initialized), no attempt is made to produce a virtual machine dump for the indicated server. System Action: SSL processing continues. System Programmer Response: None. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message DTCSSL0216W is added for Level(s): 620 610 540 DTCSSL0216W Server VMDUMP initiated by <userid> Explanation: This message is directed to the console of a TCP/IP stack or DCSS Agent server, and indicates that VMDUMP processing has been initiated by the indicated SSL server. System Action: A virtual machine dump is created for the subject machine, after which server operations continue. System Programmer Response: Collect and process the dump that is created, save any existing problem information, and contact the IBM support center for assistance. --------------------------------------------------------------- Chapter 18. SSL Messages Section: SSLADMIN, SSLIDCSS and VMSSL Messages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message DTCSSL2441E is added for Level(s): 620 610 540 DTCSSL2441E <userid> requires <command> command authorization when operand <operand> is used System Action: Command processing stops. Explanation: When the listed operand is specified, the subject server must have appropriate authorization (based on privilege class) to use a non-general version of the command cited in the message. The server currently does not have the required privilege class. System Programmer Response: Assign the necessary privilege class to the server. If necessary, consult the appropriate command documentation for information about the privilege classes that pertain to the listed command. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message DTCSSL2413E is added for Level(s): 540 DTCSSL2413E A specific target server (or, ALL) must be designated Explanation: As reported by message DTCSSL2429I, multiple servers are in operation. The supplied command is potentially disruptive and might adversely affect the operation of these servers, in addition to any secure connections that are being managed among them. To prevent inadvertent or unintended results, the subject command must be directed to only one server, all active servers, or to a predetermined subset of such servers. System Action: Command processing stops. System Programmer Response: When the subject command again is issued, include the SSLSERVER command option and designate a single server or all servers (via use of the ALL keyword) as the intended recipient(s). Alternately, to direct the command to a subset of active servers, first use the SSLADMIN SET SSLSERVER command to establish the intended recipients. Then, reissue the command as is (do not include the SSLSERVER option). --------------------------------------------------------------- Chapter 20. TCP/IP Utilities Section: TCPRUN Messages - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message 1047 is added for Level(s): 620 610 540 1047 Server <userid> is not operational Explanation: The user ID listed in the message has been determined to not be running, or in an expected state. More specific information about the state of this server is provided by one or more accompanying messages. Severity: Warning. System Programmer Response: Review accompanying messages and their documentation more information about this problem, and possible actions for its resolution. Take appropriate actions to start or restart the subject server. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message 1048 is added for Level(s): 620 610 540 1048 SLVL service information for: <file> Severity: Informational Explanation: Service information, intrinsic to the indicated file, is reported by this message for potential diagnostic use. This information is acquired and produced by the TCPSLVL command. System Programmer Response: No action is required. However, ensure this service information is included with other documentation, when problems with the subject server are diagnosed in consultation with IBM support center. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Message 1008 is added for Level(s): 610 540 1008 Server (class) <name> is not defined in file(s): <file_list> Explanation: The indicated server user ID or class was not located in any of the indicated DTCPARMS files. User Response: If a locally defined user ID is being used, ensure that a :Type.Server entry has been created to define the server in one of the listed DTCPARMS files. If a reference is being made to a locally defined class, then that class must likewise be defined in one such file. If the server was provided by IBM, ensure that any requisite service to VM TCP/IP has been applied and that all installation steps have been performed. TEMPORARY FIX: COMMENTS: MODULES/MACROS: DTCUME DTCUMEB SSLADMIN SSLADMNP SSLDPUMP SSLGSKCF SSLREPRT SSLSCBEX SSLSTART TCPRUN VMSSL SRLS: SC24612505 SC24623801 SC24623803 RTN CODES: CIRCUMVENTION: MESSAGE TO SUBMITTER: