[ofa-general] Direct / "Crossover" IPOIB

[Robert Dunkley]

Thanks Or,

I have installed and run opensm on one of the machines, I can now move
traffic between them.

Pings show a latency og 0.07-0.09ms which seems OK.

Qperf shows abysmal results in connected mode though:
[root at mrhappy OFED-1.3.1]# qperf 192.168.10.161 tcp_bw tcp_lat
tcp_bw:
    bw  =  32.8 KB/sec
tcp_lat:
    latency  =  42.2 us

Datagram mode shows better bandwidth but still awful latency:
[root at mrtickle OFED-1.3.1]# qperf 192.168.10.160 tcp_bw tcp_lat
tcp_bw:
    bw  =  400 MB/sec
tcp_lat:
    latency  =  41.2 us

Is there something wrong with my setup? (QPerf seems to think so)

Thanks,

Rob

-----Original Message-----
From: Or Gerlitz [mailto:ogerlitz at voltaire.com] 
Sent: 24 July 2008 09:44
To: Robert Dunkley
Cc: general at lists.openfabrics.org
Subject: Re: [ofa-general] Direct / "Crossover" IPOIB

Robert Dunkley wrote:
>
> I've just setup two machines linked by a single Inifiniband cable. The

> IPOIB side of things looks OK, ifconfig shows the interface and each 
> machine can ping itself on its Infiniband IP. They cannot ping each 
> other though and ibstat looks wrong. What do I need to do to make this

> setup work?
>
> Port 1:
>
> State: Initializing
>
> Physical state: LinkUp
>
> Base lid: 0
>
> SM lid: 0
>
you have to run SM (Subnet Manager), if you issue

$ opensm

on one of the nodes, it will do that.

Or.


The SAQ Group
Registered Office: 18 Chapel Street, Petersfield, Hampshire. GU32 3DZ
SEMTEC Limited trading as SAQ is Registered in England & Wales
Company Number: 06481952

http://www.saqnet.co.uk AS29219
SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business.
DSL : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support.

Find us in http://www.thebestof.co.uk/petersfield