[ofa-general] ***SPAM*** IBGD/OFED configuration/performance question

[Old Apple]

We have a CISCO/Topspin 7000D DDR switch with a
built-in subnet manager. In a minimal configuration we
need to connect a DDN 9550 disk system (one controller
with two SDR ports) and just one server node with
CISCO Tavor DDR HCA. I.e. we have two DDN outlets and
one HCA all connected to the same 7000D switch. 

Inside DDN, we have configured 6 two-tier LUNs; first
three of them are zoned to one SDR outlet, and the
remaining three - to another SDR outlet. Thus we have
6 
logical drives and must be able to see them all inside
  our only host. 

With OFED 1.2.5.4, everything connects without any
problem, all 6 disks are seen inside the server. By
setting max_sect to 16384 and srp_sg_tablesize to 256
we however obtain only 110 MB/sec per LUN which is too
low for our DDN system. (To make sure we are not
simply piping everything into one SDR port of DDN, we
tried to use only 3 LUNS together, and still never
went over 110 MB/sec per LUN). 

Exactly the same configuration with IBGD 1.8.3 loaded
on the server with max_xfer_sectors_per_io set to 8192
leads us to 220+ MB/sec per LUN, but we only see three
luns out of 6. The IBGD simply ignores one of the two
DDN ports. If we put all 6 LUNS on the only visible
port of DDN, the disks are all present, but then we
end up eating all the bandwidth of the SDR port, and
hence may leave nothing to other servers. 

To conclude, we are now trying to find out what's
wrong with OFED 1.2.5.4 performance, and/or what could
be done with IBGD 1.8.3 to ensure that our host sees
both DDN ports. 

If by chance somebody on this list had similar issues
in the past, please comment. Thanks ahead!


















      ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs