[openib-general] Howto setup rping and krping for OpenIB with AMSO1100

Thu May 18 17:26:53 PDT 2006

Hello,

After spending some time installing OpenIB with AMSO1100, I have managed
to get rping and krping to work.  Here is a howto document describing
the steps I took in the process.   This does not get opensm to work, as
I am still getting a port guid error on that front.  You are welcome to
add this to any documentation area if you find it useful.  I have not
written a detailed howto like this before, and I am also new to this
technology, so forgive me if there are any errors.

Cheers,

Chris Kasten

------------------------------------------------------------------------
---

How to install OpenIB with an Ammasso1100 network card

Enables rping, krping

Based on the version from May 10, 2006.

------------------------------------------------------------------------
---

Follow the Installation Cheat Sheet with a few variations detailed below
(substituting Ammasso for Mellanox):

https://openib.org/tiki/tiki-index.php?page=Installation+Cheat+Sheet

___________

Step 3:

-----------

I used linux kernel 2.6.16.15

In make menuconfig include the following as modules-

Device Drivers -> Infiniband support -> <M> Infiniband support

                              <M> Infiniband userspace MAD support

                              <M> Infiniband userspace access (verbs &
cm)

                              <M> Ammasso 1100 HCA support

                              <M> Kernel RDMA Ping Module

______________

Steps 5/6:

--------------

Ignore ib_mthca on modprobe

______________________________

ICS Building management tools:

------------------------------

In src/userspace/management/osm/

      rename authors -> AUTHORS

                news -> NEWS

              readme -> README

      then run ./autogen.sh && ./configure

___________________________________

Building userspace verbs libraries:

-----------------------------------

Install sysfsutils before you build these libraries.  I used
sysfsutils-1.3.0

./configure

make

make install

Instead of libmthca, build libamso

For rping, build librdmacm

Use the standard:

./autogen.sh

./configure

make

make install

for these libraries.

_______________________________________________

Don't bother with the "Testing" step and beyond

-----------------------------------------------

------------------------------------------------------------------------
-----

Download and install:

http://www.opengridcomputing.com/downloads/ogc_amso_kit_20060308.tgz.

------------------------------------------------------------------------
-----

On one of my machines, the following was not added to
/etc/modprobe.conf.dist:

(as one line)

install ib_core for i in ib_core ib_mad ib_cm ib_sa ib_ucm ib_umad
ib_uverbs; do /sbin/modprobe

      --ignore-install $i; done

Make sure this is in /etc/modprobe.conf.dist so the modules will load
automatically at boot

------------------------------------------------------------------------
-----

Check to see that the following modules are loaded:

ib_uverbs - - <-uverbs adds /dev/infiniband/uverbs0

rdma_cm

iw_cm

ib_addr

ib_cm

ib_local_sa

ib_sa

ib_core

iw_c2 - - - - <-amso

-----These two didn't load automatically for me------

rdma_krping - <-krping

rdma_ucm- - - <-rping adds /dev/infiniband/rdma_cm

------------------------------------------------------------------------
-----

Start up interfaces

-------------------

After a fresh reboot, check which modules are loaded.  I had to add
these two.

modprobe rdma_krping

modprobe rdma_ucm

Machine 1: ifconfig iw0 192.168.69.149 up

(server)   ifconfig eth0 192.168.68.149 up

Machine 2: ifconfig iw1 192.168.69.148 up

(client)   ifconfig eth1 192.168.68.148 up

------------------------------------------------------------------------
-----

Testing:

______

krping

------

On server: /bin/echo server,port=9999,addr=192.168.69.149,validate >
/proc/krping

On client: /bin/echo client,port=9999,addr=192.168.69.149,validate >
/proc/krping

(addr = server iw addr)

To verify: cat /proc/krping

1 listen: indicates the server is waiting

1-amso0 n1 n2 n3 ...: indicates the connection has been made

If n = 0 for all n, there is probably an error

_____

rping

-----

Server: rping -s -vV -C10 -S10 -a 0.0.0.0 -p 9999

Client: rping -c -vV -C10 -S10 -a 192.168.69.149 -p 9999

This should output:

ping data : rdma-ping (10x because of -C10)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openfabrics.org/pipermail/general/attachments/20060518/5bce40da/attachment.html>