<!doctype html public "-//W3C//DTD W3 HTML//EN">
<html><head><style type="text/css"><!--
blockquote, dl, ul, ol, li { padding-top: 0 ; padding-bottom: 0 }
--></style><title>RE: [openib-general] ip over ib
throughtput</title></head><body>
<div>First, I love Hardware Reliability.</div>
<div><br></div>
<div>1QP per node, this might be fine for small clusters, but what
about larger clusters, where I have an all-to-all communications
pattern ? What about say *IF* something like IB was ever designed into
a BG/L class cluster ? What about if I happen to want to run a
multi-protocol application, how is that currently handled ? *IF* I am
allowed to do that and I had an all-2-all communication pattern, looks
like I would exhaust the QP allocation table. Sounds like potentially,
as clusters/nodes increase in number, we could easily bump our heads
on some of the IBTA artificial limits set. IMHO</div>
<div><br></div>
<div>Instead of 1 QP for every node, don't you mean 1 QP for every
HCA/Node ? That assumes that I can have multiple HCAs per node.
Currently even at the IPoIB level, we can not do
"bonding/trunking", correct ?</div>
<div><br></div>
<div>Steve...</div>
<div><br></div>
<div>At 12:11 PM -0800 1/5/05, Diego Crupnicoff wrote:</div>
<blockquote type="cite" cite><font size="-1">> -----Original
Message-----</font><br>
<font size="-1">> From: Woodruff, Robert J [</font><a
href="mailto:robert.j.woodruff@intel.com"><font
size="-1">mailto:robert.j.woodruff@intel.com</font></a><font
size="-1">]</font><br>
<font size="-1">> Sent: Wednesday, January 05, 2005 3:53
PM</font><br>
<font size="-1">> To: Diego Crupnicoff; Hal Rosenstock; Peter
Buckingham</font><br>
<font size="-1">> Cc: openib-general@openib.org</font><br>
<font size="-1">> Subject: RE: [openib-general] ip over ib
throughtput</font><br>
<font size="-1">></font><br>
<font size="-1">></font><br>
<font size="-1">> The trade off is that a fully connected model
requires each</font><br>
<font size="-1">> node to burn a QP for every node in the cluster
and thus does</font><br>
<font size="-1">> not scale as well as the UD model. My guess
is that if</font><br>
<font size="-1">> people need really high performance</font><br>
<font size="-1">> socket access, they will use SDP
instead.</font><br>
<font size="-1">></font><br>
<font size="-1">> woody</font><br>
<font size="-1">></font><br>
</blockquote>
<blockquote type="cite" cite><font size="-1">1 QP for every node in
the cluster does not sound that bad.</font><br>
</blockquote>
<blockquote type="cite" cite><font size="-1">SDP is a good alternative
too. It has even further benefits as compared to IPoIB (built in HW
reliability that eliminates the TCP/IP stack, potential for zero copy,
etc). However, in terms of QP requirements, SDP would consume even
more than what a connected mode IPoIB would (still not too bad given
the IB HW capabilities).</font><br>
</blockquote>
<blockquote type="cite" cite><font size="-1">Diego</font><br>
</blockquote>
<blockquote type="cite" cite><br>
<br>
</blockquote>
<blockquote type="cite" cite><br>
_______________________________________________<br>
openib-general mailing list<br>
openib-general@openib.org<br>
http://openib.org/mailman/listinfo/openib-general<br>
<br>
To unsubscribe, please visit
http://openib.org/mailman/listinfo/openib-general</blockquote>
<div><br></div>
<div><br></div>
<x-sigsep><pre>--
</pre></x-sigsep>
<div>Steve Poole (spoole@lanl.gov)<x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab>Office: 505.665.9662<br>
Los Alamos National Laboratory<x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab>Cell: 505.699.3807<br>
CCN - Special Projects / Advanced
Development<x-tab>
</x-tab><x-tab>
</x-tab><x-tab>
</x-tab>Fax: 505.665.7793<br>
P.O. Box 1663, MS B255<br>
Los Alamos, NM. 87545<br>
03149801S</div>
</body>
</html>