[ofa-general] low performance with multiple LUNs on a single port with ib_srp

[Canon, Richard Shane]

Greetings,

 

Hopefully the subject says it all...

 

I've stumbled on a performance issue with the OFED ib_srp driver.  Here
is the configuration.  I am testing with a DDN 9550 and a single host
system.  The systems are connected by two SDR links.  On the host side
there is a dual port (DDR) card.  On the DDN side, both lines go into a
single singlet (even though it is a couplet).  The lines go into two
distinct cards on the DDN side (if you are familiar with the layout).
The testing used OFED 1.2.

 

Now for the tests...  If I run a single stream test I'm seeing good
result with over 700 MB/s.  These tests are run using sg_dd with the
directio flag.  If I run two concurrent streams against two LUNs that
are each presented over a single port on the DDN (and therefore accessed
by a single port on the host side), the aggregate performance drop to
around 120 MB/s (60 MB/s per stream).

 

Just to confirm it isn't a problem on the DDN side, I repeated these
tests with the IBGD driver.  There I consistently saw about 600-650 MB/s
on the port regardless of the number of LUNs I tested with.

 

Any ideas on what the problem is?  Also, if this doesn't make sense, let
me know and I will try to clarify further.

 

Thanks,

 

--Shane Canon

 

 

 

--

R. Shane Canon

National Center for Computational Science

Oak Ridge National Laboratory

canonrs at ornl.gov

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20070625/0b2223b1/attachment.html>