[ofa-general] Setting the rate in Infiniband.

[Roland Dreier]

 > The reason why I need such small rates is because I interface the Infiniband
 > HCA to an FPGA via an Infiniband physical link.  Imagine the FPGA as a
 > simple repeater that simply forwards the infiniband signals to the Target
 > HCA. The FPGA cannot handle such a high data rate and neither do I have as
 > much memory as required to buffer it on the FPGA (I might drop packets if
 > the buffer becomes full). Hence I wish to limit the rate to say 100Mbps
 > instead of 2.5Gbps.
 > 
 > Question:-
 > How can I set rates less than 2.5Gbps? Can this be changed at all?

The IB physical layer does not define any signaling slower than 2.5Gbps,
ie 1 lane of single data rate.  There is nothing slower than any real IB
HCA or switch will be able to do.  Therefore for your FPGA to be able to
talk IB at all, you will need to be able to handle a 1X SDR link (and do
8b/10b encoding, etc).  Note that the 8b/10b encoding means there is
really only 2 Gbps of data on a 1X SDR link.

However, it is OK (in theory) for your FPGA to handle only a the minimal
IB MTU (256 bytes), and it also OK for your FPGA to give only a small
number of link-layer credits (whatever it has buffering for).  This
should limit the data you have to buffer to what you can handle, and
lets you throttle the traffic at the link level.  You will still need to
be able to handle link packets, idles, etc at the full data rate.

With that said, I would expect any FPGA fancy enough to have a SERDES
capable of doing IB signaling to be able handle 2 Gbps of real traffic,
since I've seen designs doing fairly complex processing using Virtex II
(ie 5+ year old FPGAs) able to handle full 4X SDR IB links.  I guess it
depends on the sophistication of your RTL.

 - R.