[ofa-general] Bandwidth of performance with multirail IB

[Jie Cai]

Davis, Arlin R wrote:
>  
> Jie, 
>
>   
>> A question on implementation multi-rail with uDAPL connections.
>>
>> What I did is open 2 IAs (corresponding to the 2 ports on 
>> HCAs) on each 
>> node.
>> Then create one EP for each IA, and connect those EPs to the 
>> corresponding
>> EP at other node.
>>
>> Then data been transferred via both EP-connections.
>>     
>
> Did you verify data flow is across independent HCA channels
> using perfquery?
>   

I have using perfquery to verified that data has been transmit
through both channel. The ourput for perfquery is:

############# Before data transfer: #############
# Port counters: Lid 8 port 1
PortSelect:......................1
CounterSelect:...................0x0000
LinkDowned:......................2
XmtDiscards:.....................39
XmtData:.........................4294967295
RcvData:.........................20456971
XmtPkts:.........................112151685
RcvPkts:.........................2372783

# Port counters: Lid 9 port 2
PortSelect:......................2
CounterSelect:...................0x0000
LinkDowned:......................1
XmtDiscards:.....................21
XmtData:.........................4294967295
RcvData:.........................7123011
XmtPkts:.........................62927413
RcvPkts:.........................896106

############# After data transfer: #############

# Port counters: Lid 8 port 1
PortSelect:......................1
CounterSelect:...................0x0000
LinkDowned:......................2
XmtDiscards:.....................39
XmtData:.........................4294967295
RcvData:.........................20457865
XmtPkts:.........................112168095
RcvPkts:.........................2372882

# Port counters: Lid 9 port 2
PortSelect:......................2
CounterSelect:...................0x0000
LinkDowned:......................1
XmtDiscards:.....................21
XmtData:.........................4294967295
RcvData:.........................7123912
XmtPkts:.........................62943824
RcvPkts:.........................896206

Difference in XmtPkts of port1  is 16410, and the difference for port 2 
is 16411.

Does this suggest that around similar size of data has been transfered
through both rails?

>   
>> I have been notice that there's a MULTIPATH connection flag for dapl, 
>> but I did
>> not use it. What's the use of it?
>>     
>
> Multipath is not supported at this time.
>
> I will roll up a uDAPL test and see if I can verify performance.
>
> -arlin
>
>