[openib-general] ip over ib throughtput

[Stephen Poole]

First, I love Hardware Reliability.

1QP per node, this might be fine for small clusters, but what about 
larger clusters, where I have an all-to-all communications pattern ? 
What about say *IF* something like IB was ever designed into a BG/L 
class cluster ? What about if I happen to want to run a 
multi-protocol application, how is that currently handled ? *IF* I am 
allowed to do that and I had an all-2-all communication pattern, 
looks like I would exhaust the QP allocation table. Sounds like 
potentially, as clusters/nodes increase in number, we could easily 
bump our heads on some of the IBTA artificial limits set. IMHO

Instead of 1 QP for every node, don't you mean 1 QP for every 
HCA/Node ? That assumes that I can have multiple HCAs per node. 
Currently even at the IPoIB level, we can not do "bonding/trunking", 
correct ?

Steve...

At 12:11 PM -0800 1/5/05, Diego Crupnicoff wrote:
>  > -----Original Message-----
>>  From: Woodruff, Robert J 
>>[<mailto:robert.j.woodruff at intel.com>mailto:robert.j.woodruff at intel.com]
>>  Sent: Wednesday, January 05, 2005 3:53 PM
>>  To: Diego Crupnicoff; Hal Rosenstock; Peter Buckingham
>>  Cc: openib-general at openib.org
>>  Subject: RE: [openib-general] ip over ib throughtput
>>
>>
>>  The trade off is that a fully connected model requires each
>>  node to burn a QP for every node in the cluster and thus does
>>  not scale as well as the UD model.  My guess is that if
>>  people need really high performance
>>  socket access, they will use SDP instead.
>>
>>  woody
>>
>
>1 QP for every node in the cluster does not sound that bad.
>
>SDP is a good alternative too. It has even further benefits as 
>compared to IPoIB (built in HW reliability that eliminates the 
>TCP/IP stack, potential for zero copy, etc). However, in terms of QP 
>requirements, SDP would consume even more than what a connected mode 
>IPoIB would (still not too bad given the IB HW capabilities).
>
>Diego
>
>
>
>
>_______________________________________________
>openib-general mailing list
>openib-general at openib.org
>http://openib.org/mailman/listinfo/openib-general
>
>To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general


-- 
Steve Poole (spoole at lanl.gov) 
	Office: 505.665.9662
Los Alamos National Laboratory					Cell: 
505.699.3807
CCN - Special Projects / Advanced Development			Fax: 
505.665.7793
P.O. Box 1663, MS B255
Los Alamos, NM. 87545
03149801S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20050106/538f4a1c/attachment.html>