[ofa-general] Tools for Infiniband Performance measurement

[Jeff Squyres]

Note that NetPIPE 3.7 will only use the first port of the first HCA  
in your machine (which is a fairly common configuration).

I have submitted a patch to the NetPIPE maintainers that allows you  
to specify on the command which port / HCA to use for testing.



On Mar 20, 2007, at 5:43 PM, Kyle Schochenmaier wrote:

> Dukle, Kapil (GE Healthcare) wrote:
>> Hi all,
>>  I was wondering if there are any tools specific for Infiniband- 
>> performance measurement - throughput, round trip time with adjustable
>> packet sizes.
>>
> The latest NetPIPE benchmarking utility has Infiniband support:
> It will measure what you asked.
>
> http://source.scl.ameslab.gov/NetPIPE/NetPIPE_3.7.tar.gz
>
> Kyle
>> --------------------------------------------------------------------- 
>> ---
>> *From:* Dukle, Kapil (GE Healthcare)
>> *Sent:* Monday, March 19, 2007 5:37 PM
>> *To:* general at lists.openfabrics.org; openib-general at openib.org
>> *Subject:* ibping fails in loopback mode
>>
>> Hi all,
>>  I'm having trouble getting ibping to work in loopback node. The  
>> original config is to connect first port of Blade 1 with first  
>> port of Blade 2.
>> For my loopback test, I have connected the Infiniband cable  
>> between the 2 ports on the same Linux blade.
>> Both opensm and ibping (server mode) seem to be running on the  
>> blade. Ping succeeds to the first active port, but fails
>> for the second one. See below...
>>  Why is the ibping to the second active (and linkup) port  
>> failing?  Am I missing something? The version is OFED1.0
>>   *[root at XXXX]# ps -elf | grep opensm*
>> 4 S root      3078     1  0  76   0 - 19039 stext  Mar16 ?         
>> 00:00:09 */usr/sbin/opensm*
>> 4 S root      5444  5424  0  77   0 - 13981 pipe_w 17:22 pts/1     
>> 00:00:00 grep opensm
>> *[root at XXXX]# ibping -v -S &
>> *[1] 5445
>> [root at vre sdc]# ibwarn: [5445] ibping_serv: starting to serve...
>>  *[root at XXXX]# ps -elf | grep ibping*
>> 4 S root      5445  5424  0  77   0 -  1454 -      17:22 pts/1     
>> 00:00:00 *ibping -v -S*
>> 4 S root      5447  5424  0  77   0 - 13982 -      17:22 pts/1     
>> 00:00:00 grep ibping
>>  *[root at XXXX]# sminfo*
>> *sminfo: sm lid 0x3* sm guid 0x3ba00010027d9, activity count  
>> 203459 priority 1 state SMINFO_MASTER 3
>>  *[root at XXXX]# ibstat*
>> CA 'mthca0'
>>         CA type: MT25208 (MT23108 compat mode)
>>         Number of ports: 2
>>         Firmware version: 4.7.400
>>         Hardware version: a0
>>         Node GUID: 0x0003ba00010027d8
>>         System image GUID: 0x0003ba00010027db
>>         *Port 1:
>>                 State: Active
>>                 Physical state: LinkUp
>>                 Rate: 10
>>                 Base lid: 3*
>>                 LMC: 0
>>                 SM lid: 3
>>                 Capability mask: 0x02510a6a
>>                 Port GUID: 0x0003ba00010027d9
>>         *Port 2:
>>                 State: Active
>>                 Physical state: LinkUp
>>                 Rate: 10
>>                 Base lid: 1*
>>                 LMC: 0
>>                 SM lid: 3
>>                 Capability mask: 0x02510a68
>>                 Port GUID: 0x0003ba00010027da
>>  *[root at XXXX]# ibping -v 3*
>> ibwarn: [5452] ibping: Ping..
>> ibwarn: [5445] ibping_serv: Pong: vre.(none)
>> *Pong from vre.(none) (Lid 0x3): time 0.111 ms*
>> ibwarn: [5452] ibping: Ping..
>> ibwarn: [5445] ibping_serv: Pong: vre.(none)
>> Pong from vre.(none) (Lid 0x3): time 0.087 ms
>> ibwarn: [5452] ibping: Ping..
>> ibwarn: [5445] ibping_serv: Pong: vre.(none)
>> Pong from vre.(none) (Lid 0x3): time 0.069 ms
>> ibwarn: [5452] report: out due signal 2
>>  --- vre.(none) (Lid 0x3) ibping statistics ---
>> *3 packets transmitted, 3 received, 0% packet loss, time 2320 ms*
>> rtt min/avg/max = 0.069/0.089/0.111 ms
>> *[root at XXXX]# ibping -v 1*
>> ibwarn: [5449] ibping: Ping..
>> *ibwarn: [5449] main: ibping to Lid 0x1 failed*
>> ibwarn: [5449] ibping: Ping..
>> ibwarn: [5449] main: ibping to Lid 0x1 failed
>> ibwarn: [5449] ibping: Ping..
>> ibwarn: [5449] main: ibping to Lid 0x1 failed
>> ibwarn: [5449] report: out due signal 2
>>  ---  (Lid 0x1) ibping statistics ---
>> *3 packets transmitted, 0 received, 100% packet loss, time 11358 ms*
>> rtt min/avg/max = 0.000/0.000/0.000 ms
>>  Thanks,
>> Kapil
>> !DSPAM:46003995205571657414402!
>> --------------------------------------------------------------------- 
>> ---
>>
>> _______________________________________________
>> general mailing list
>> general at lists.openfabrics.org
>> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>>
>> To unsubscribe, please visit http://openib.org/mailman/listinfo/ 
>> openib-general
>>
>> !DSPAM:46003995205571657414402!
>>
>
>
> -- 
> Kyle Schochenmaier
> kschoche at scl.ameslab.gov
> Research Assistant, Dr. Brett Bode
> AmesLab - US Dept.Energy
> Scalable Computing Laboratory
> _______________________________________________
> general mailing list
> general at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/ 
> openib-general


-- 
Jeff Squyres
Cisco Systems