[ewg] Capturing infiniband packets on the wire

[Julien Lafaye]

On Tue, Apr 5, 2011 at 3:39 PM, Greg Kerr <kerr.g at husky.neu.edu> wrote:
>>  There is NO outbound frame in my ibdump captures:
>>   - is that expected behavior ?
>
> When you say outbound frame... specifically what packets are you seeing in
> Wireshark? It should show what 'type' each packet is. Are you not seeing any
> packets labeled 'RC Send First'? Are you only seeing packets labeled 'RC Recv
> ...'?

I am attaching the pcap files to this email.

You are right, it seems that both inbound and outbound frames are
found in the two capture files. In wireshark (I use wireshark for
Windows 1.4.4 by the way), I have inconsistencies in the displayed
local ids:
- on host-a.pcap: all frames are displayed as having destination local
id: 14 and source local id: 15
- on host-a.pcap: all frames are displayed as having destination local
id: 15 and source local id: 14

This is what made me think that only outbound frames were captured. I
guess this is a bug in the dump tool (ibdump) from Mellanox. Can
someone confirm this ?

>>   - as far as I have understood, the queue pair to which payloads are
>>   sent is coded in the IB frame. How come can a program observe frames
>>   sent to an other queue pair using its own queue pair ?
>
> The destination QP is coded in the Base Transport Header, specified on page 155
> of the current specification.
>
> I'm not sure I understand the rest of your question. Are you suggesting that
> you want to write a program to do such a thing, or are you asking how it is
> ibdump can display all packets?

As many socket programmers I have found very convenient to use tcpdump
for capturing and replaying network flow. I tried to use ibdump for
that purpose. From the observation below it seems like this tool does
the job except for local ids. So I am questioning whether it is
possible to code such a tool using native verbs functionalities
(writing a packet capture utility usually consists only on opening a
RAW socket).

>>   - is ibdump relying on Mellanox card specific features ? Is source
>>   code available ?
>
> I'm not sure, but I assume the source code is closed and that it rely on card
> specific (or at least driver specific) features. It only runs on Mellanox
> ConnectX®/ConnectX®-2 adapters.

I guess I should contact Mellanox support directly. I think my company
has a contracted support with them.

> --
> Greg Kerr
> High Performance Computing Lab
> Northeastern University
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: host-a.pcap
Type: application/octet-stream
Size: 44486 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20110405/cdc2bfd8/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: host-b.pcap
Type: application/octet-stream
Size: 44824 bytes
Desc: not available
URL: <http://lists.openfabrics.org/pipermail/ewg/attachments/20110405/cdc2bfd8/attachment-0001.obj>