[openib-general] IBM utilizing and testing openIB release 1 RCx

[Kevin Reilly]

   We wanted to let others in the OpenIB community know that IBM is using
and testing openIB as it's base support for it's IBM HCA
as part of  linux on Power offerings.  We are in the process of testing
much of our clustering  software and high performance subsystems
layered over some of the openIB interfaces like the user verbs and
IPoverIB.

     So far the testing has been going very well and most of the problems
we hit were found and fixed soon after we hit them.
Many times the fix was already  in a newer subversion level when we hit it.
As we hit problems we'll open bugzillas and we will
report  problems so they can be picked up in newer RC levels.

  The biggest problem we have at the moment is being able to tune the
IPoverIB drivers  sendq_size and recvq_size.  We simply
can get IP to work without tuning these parameters in some situations.  For
example, running our cluster filesystem file server over
IPoverIB.  The server that is a filesystem server needs to have many socket
connections open and we use a big 8 or 16 way server
for this purpose. We very quickly overrun the IPoverIB drivers send and
receive queue. We are submitting a patch so we can tune
these because we couldn't get things to work with the defaults and not way
to tune.

Kevin J. Reilly
STSM, HPC Architecture
-HPC interconnect architect
(office) 845-433-7976  (tieline) 8-293-7976