|
TCP/IP Enhancements on PUT 14: More
Tools to Get the Job Done
David Parker, Rick Schoonmaker, and Jamie
Farmer, IBM TPF Development
With the increasing use of TPF TCP/IP native
stack and Open Systems Adapter (OSA)-Express support, it has become necessary
to develop some new tools that will greatly benefit TCP/IP application
programmers, operators, and other end users. On TPF PUT 14, we enhanced
some of the TCP/IP functional messages, developed two new functional messages,
and provided some new functions for TCP/IP application programmers.
"Now you see it, now you don't . . ."
APAR PJ27451
TPF TCP/IP native stack support provided
the ZTTCP functional message to manipulate (activate, change, define, delete,
display, inactivate, and trace) IP links in the TPF system. After going
through our internal testing cycle on both the original native stack support
(IP over CDLC) and OSA-Express, and using some of the display functions
provided by TPF native stack support, we saw an opportunity to enhance
some of the display functions.
The first opportunity was for two simple
enhancements to the ZTTCP DISPLAY functional message with the STATS parameter
specified. This functional message was created to display statistics and
resources used by TCP/IP native stack support. With APAR PJ27451, TPF now
provides the ability to reset or clear the statistic counters with the
CLEAR parameter on the ZTTCP functional message. This will give you the
ability to start monitoring your TCP/IP network whenever you choose. There
was also a need to add a new line to the ZTTCP DISPLAY STATS functional
message. At times, an application would issue a socket send() API and the
TCP/IP stack would send the message to the network. If the message did
not get acknowledged, we retransmitted the message and eventually cleaned
up the socket if the remote end did not acknowledge our message. However,
it was difficult to determine why the socket was getting cleaned up from
the application point of view. Therefore, a line was added to the ZTTCP
DISPLAY STATS functional message display for the number of TCP sockets
that have been cleaned up because of retransmit timeouts.
The second enhancement was to the ZTTCP
DISPLAY functional message with the ALL parameter specified. Before the
release of APAR PJ27451, this functional message did not display the symbolic
device addresses (SDAs) associated with the OSA-Express connections. Not
knowing which SDAs were mapped to your OSA-Express connections and having
to enter multiple functional messages to acquire that information was not
efficient. The ZTTCP DISPLAY functional message with the ALL parameter
specified now displays the SDAs associated with each OSA connection defined
in the TPF system.
On PUT 13, OSA-Express support (APAR PJ27333)
was added to the TPF system. Included with OSA-Express support was movable
virtual IP address (VIPA) support, which gave users the ability to move
virtual IP addresses from one processor to another in a loosely coupled
environment to assist in load balancing TCP/IP connections. A movable VIPA
must be defined to all the different processors that it can move to. The
ZVIPA functional message with the DISPLAY and IP parameters specified allowed
you to display VIPAs defined to the TPF system. However, there was no way
to display all the CPUs to which a specific VIPA was defined without displaying
the VIPA on each individual processor. With APAR PJ27451, the ZVIPA functional
message with the DISPLAY and IP parameters specified will now display all
the CPU IDs to which the movable VIPA is defined.
"Balance is the key . . ."
APAR PJ27491
As stated previously, TPF OSA-Express support
on PUT 13 gave users the ability to move virtual IP addresses from one
processor to another in a loosely coupled environment. This support was
added to help balance TCP/IP workload. The original VIPA support only allowed
users to move a VIPA by issuing the ZVIPA functional message with the MOVE
parameter specified or by using the UVIP user exit, which is entered when
a processor is deactivated. Just imagine if you had the ability to write
some logic in an application to have the application issue the command
to move a VIPA to the processor of its choice to load balance TCP/IP connections.
You then would not have to rely solely on operators to assist in the balancing
of TCP/IP workloads in the TPF system. The necessary logic could be coded
into a user application that would allow it to do the moving of VIPAs.
With APAR PJ27491, this is a reality. Two new program interfaces have been
created to allow application programs to move VIPAs by using either an
assembler macro called VIPAC or a C/C++ function called tpf_vipac( ).
"Life is a balancing act . . ."
APAR PJ27679
Staying with the topic of balancing TCP/IP
workload, another useful function has been provided for user applications.
In the past, ECBs created by the activate_on_receipt( ) and activate_on_receipt_with_length(
) socket APIs were created on the same I-stream in which the AOR was issued
to support applications that can only run on one I-stream. Many of today's
applications are able to run on any I-stream, so it would be nice to have
the ability to create these new ECBs on any I-stream as well. With APAR
PJ27679, applications that use the activate_on_receipt( ) or activate_on_receipt_with_length(
) socket APIs can now choose to have the ECBs created from these socket
APIs created on the least busy I-stream. An ioctl( ) option called AOR_BALANCE
was added to help load balance ECBs created by the AOR socket APIs. With
the AOR_BALANCE ioctl( ) option set on by a socket, new ECBs created by
AOR for that socket will be created on the least busy I-stream rather than
on the I-stream in which the AOR was issued.
"IP trace moves pretty fast; if you
don't stop and look, you could miss it."
APAR PJ27617
Having good TCP/IP diagnostic tools is
very important when it comes to solving connection problems. The original
native stack support provided users with IP trace capability. When you
have thousands of TCP/IP connections and you are having problems with only
one connection, you don't want to filter through hundreds of traced IP
packets offline to find the two or three packets related to that one connection
that is having problems. If a problem occurs with one TCP/IP connection
on a production system, it is inconvenient to trace packets being sent
and received for a connection by using the offline trace. Because of the
amount of traffic on a production system, the trace would wrap before the
packets were able to be viewed.
To make gathering trace data for a particular
connection easier and to provide a more viable trace tool for performance-critical
systems, APAR PJ27617 has added individual IP trace support. A new functional
message, ZINIP, has been added to the TPF TCP/IP diagnostic tools portfolio.
The ZINIP functional message provides you with the ability to define and
manage individual IP traces on the TPF system. Individual IP trace allows
you to trace packets to and from specific nodes on the network.
Following is an example of how this individual
IP trace support may be used. In the example, a user at remote IP address
9.117.107.167 is trying to FTP something to the TPF system, but the connection
request keeps getting rejected. To determine why packets from remote IP
address 9.117.107.167 that are destined for port 21 are being rejected,
an individual IP trace will be defined. First, an individual trace is defined
to trace packets from remote IP address 9.117.107.167 that are destined
for port 21, and the trace is assigned a name, CLIENT1. The name of the
individual trace will be used to display the trace table in the future.
ZINIP DEFINE NAME-CLIENT1 RIP-9.117.107.167 PORT-21
INIP0001I 11.44.27 INDIVIDUAL IP TRACE CLIENT1 DEFINED
ZINIP SUMMARY
CSMP0097I 11.44.36 CPU-B SS-BSS SSU-HPN IS-01
INIP0005I 11.44.36 INDIVIDUAL IP TRACE SUMMARY
NAME REMOTE IP PORT SIZE WRAP PAUSED STATUS
-------- --------------- ----- ---- ---- ------ ------
CLIENT1 9.117.107.167 21 3840 YES NO EMPTY
END OF DISPLAY
The individual trace is then displayed
using the name given to the trace on the define and in a formatted display
using the FORMAT option.
ZINIP DISPLAY NAME-CLIENT1 ALL FORMAT
CSMP0097I 11.50.35 CPU-B SS-BSS SSU-HPN IS-01
INIP0007I 11.50.35 INDIVIDUAL IP FORMATTED TRACE CLIENT1 DISPLAY
RWI-02 IPCCW-D2 SOURCE IP-9.117.107.167 DEST IP-9.117.236.131 LEN-48
TOD-B5A1D03E12489447 PROTOCOL-06 (TCP) SOURCE PORT-1328 DEST PORT-21
SEQ-1324397844 WINDOW-16384 URGENT OFFSET-0
TCP FLAG BYTE-02 (SYN)
IP HEADER 45000030 1F534000 7D067360 09756BA7 0975EC83
TCP HEADER 05300015 4EF0B514 00000000 70024000 C4C40000 02040FB0 01010402
RWI-01 IPCCW-D2 SOURCE IP-9.117.236.131 DEST IP-9.117.107.167 LEN-40
TOD-B5A1D03E16349C83 PROTOCOL-06 (TCP) SOURCE PORT-21 DEST PORT-1328
SEQ-0 ACK-525549569 WINDOW-0 URGENT OFFSET-0
TCP FLAG BYTE-14 (ACK, RST)
IP HEADER 45000028 CEF00000 3C0644CB 0975EC83 09756BA7
TCP HEADER 00150530 00000000 1F534001 50140000 E0220000
From this display, we can see that the
connection request (SYN) comes in from remote IP address 9.117.107.167,
destined for port 21. We can then see TPF rejecting the connection with
a reset (RST). The common reason for this occurrence is that the server
is not active, so we will check to see if that is the case.
ZDTCP NETSTAT
CSMP0097I 14.29.27 CPU-B SS-BSS SSU-HPN IS-01
DTCP0003I 14.29.27 TPF TCP/IP NETSTAT
SOCKET TYPE LOCAL IP LPORT COUNT STATUS
-------- ---- --------------- ----- ----- ------
00C00001 UDP ANY 520 ***** SERVER
00C00004 TCP ANY 9999 9 SERVER
00C00009 TCP ANY 1414 6 SERVER
00C00014 TCP ANY 80 30 SERVER
0 RAW SOCKETS
1 UDP CLIENT SOCKETS
6 TCP CLIENT SOCKETS
56 TOTAL SOCKETS
END OF DISPLAY
From this display, the problem is that
the FTP server is not active, and that is why the connection request from
remote IP address 9.117.107.167 is being rejected. This type of problem
can now be debugged easily in real time on a production system using the
individual IP trace support.
Another enhancement that was added with
APAR PJ27617 has to do with Routing Information Protocol (RIP) messages.
With OSA-Express support, the TPF system uses RIP messages to broadcast
VIPA information to the gateways. From an IP trace perspective, these RIP
message can be annoying on a test system because all RIP messages that
are sent and received are traced by the TPF system. If you are looking
at an IP trace, it can be cumbersome trying to sort through all the RIP
messages to find the packets that are of interest. There is now an option
to trace RIP messages or to leave them out of the trace. The default option
will be to not trace the RIP messages that are sent and received
by the TPF system.
"Is there a doctor in the house . .
."
APAR PJ27650
With APAR PJ27650, TPF now has a new diagnostic
tool that will allow users to determine if a specific socket or an OSA-Express
connection is stalled. The ZSOCK and ZOSAE functional messages were enhanced
with the DATAFLOW parameter. This new function is useful when trying to
determine if data is flowing across a particular socket or OSA-Express
connection.
When the ZOSAE functional message is entered
with the DATAFLOW parameter specified, a reading is taken for a 5-second
interval, and the number of packets that were sent and received will be
displayed. In the following example, the number of packets sent and received
on all OSA-Express connections defined to the TPF system over a 5-second
interval will be displayed.
ZOSAE DATAFLOW
CSMP0097I 10.34.54 CPU-B SS-BSS SSU-HPN IS-01
OSAE0008I 10.34.54 BEGIN PROCESSING OSA DATAFLOW STATISTICS
CSMP0097I 10.34.59 CPU-B SS-BSS SSU-HPN IS-01
OSAE0009I 10.34.59 OSA DATAFLOW STATISTICS FOR A 5-SECOND INTERVAL
PACKETS PACKETS
OSA SENT RECEIVED
-------- ---------- ----------
TEST2 1423 623
TEST3 0 0
TEST4 2424 896
END OF ZOSAE DATAFLOW DISPLAY
When the ZSOCK functional message is entered
with the DATAFLOW parameter specified, a reading will be taken for a 5-second
interval and then the number of bytes sent and received will be displayed.
In the following example, we would like to see if a particular socket is
stalled. First, find the socket in question by entering the ZSOCK SUMMARY
functional message with the remote IP address. This will display all the
sockets for remote IP address 9.117.107.167. Using the socket descriptor
found on the ZSOCK SUMMARY functional message, check the data flow for
that socket. We can see that data is flowing over the 5-second interval,
so the socket is not stalled.
ZSOCK SUMMARY RIP-9.117.107.167
CSMP0097I 14.21.47 CPU-B SS-BSS SSU-HPN IS-01
SOCK0021I 14.21.47 SOCKET SUMMARY INFORMATION
SOCKET LOCAL LOCAL REMOTE REMOTE PROT STATE
DESC IP PORT IP PORT
00C00007 9.117.241.012 80 9.117.107.167 2258 TCP ESTABLISHED
SUMMARY TOTAL 1
END OF DISPLAY
ZSOCK DATAFLOW SOCKET-c00007
CSMP0097I 14.15.57 CPU-B SS-BSS SSU-HPN IS-01
SOCK0024I 14.15.57 BEGIN PROCESSING SOCKET DATAFLOW STATISTICS
CSMP0097I 14.16.02 CPU-B SS-BSS SSU-HPN IS-01
SOCK0025I 14.16.02 SOCKET DATAFLOW STATISTICS FOR A 5-SECOND INTERVAL
SOCKET DESCRIPTOR-C00007
BYTES SENT - 24538
BYTES RECEIVED - 29865
END OF ZSOCK DATAFLOW SOCKET DISPLAY
"Not as Fast, but Fast Enough"
APAR PJ27625
The original TPF OSA-Express support only
supported connectivity to Gigabit Ethernets. With APAR PJ27625, support
has been added to allow TPF to communicate with OSA-Express cards with
a Fast Ethernet adapter running in queued direct I/O (QDIO) mode. See "The
Best Fast Food in Networking," in the First Quarter 2001 issue of the TPF
Systems Technical Newsletter for more information and for examples of this
support.
TPF TCP/IP PUT 14 enhancements provide
you with new and useful tools and functions that should be very helpful
in diagnosing TCP/IP problems and balancing TCP/IP workload.
Second Quarter 2001 - Table of Contents
|
