[openib-general] Re: ANNOUNCE: First usable version of userspace verbs
Michael S. Tsirkin
mst at mellanox.co.il
Thu Feb 24 06:09:17 PST 2005
Quoting r. Roland Dreier <roland at topspin.com>:
> Subject: ANNOUNCE: First usable version of userspace verbs
>
> I'm happy to announce the initial availability of userspace verbs
> support for brave testers.
>
> To try this out, check out the roland-uverbs subversion branch:
>
> svn co https://openib.org/svn/gen2/branches/roland-uverbs
>
> and build as usual. Select CONFIG_INFINIBAND_USER_VERBS to build
> userspace verbs support.
>
> If you want to use a linux-2.6.10 kernel, you will need to apply the
> new linux-2.6.10-backports.diff patch from the branch (which just
> exports get_sb_pseudo()). No patches at all are required for an
> up-to-date BK or linux-2.6.11-rc4 tree.
>
> If you use udev, add the rule
>
> KERNEL="uverbs*", NAME="infiniband/%k", MODE="0666"
>
> to your configuration. Otherwise, create the required device files:
>
> mknod /dev/infiniband/uverbs0 c 231 128
> mknod /dev/infiniband/uverbs1 c 231 129
>
> and so on for as many HCAs as you have installed.
>
> The build the userspace libraries in src/userspace/libibverbs and
> src/userspace/libmthca with the usual
>
> ./autogen.sh && ./configure && make && sudo make install
>
> passing whatever parameters to configure you want; you can use
> --prefix to install to another location. If you set a non-standard
> prefix, it may be useful to pass a -I<path> in CPPFLAGS to the
> configure for libmthca.
>
> Once you have the libraries built and installed, load the ib_mthca and
> ib_uverbs modules. By default, libibverbs will search for driver
> libraries in <prefix>/lib/infiniband; if you installed libmthca
> somewhere else, set the OPENIB_DRIVER_PATH environment variable to
> point to the directory with mthca.so.
>
> To actually try things out, you can use the ibv_pingpong program
> shipped as part of the libibverbs package. For example, one one
> system start the server side
>
> $ ibv_pingpong
>
> and on another system start the client by passing the address of the
> server (in this example I use IPv6 over IPoIB):
>
> $ ibv_pingpong fe80::202:c901:7fc:c711%ib0
>
> The pingpong program has a number of options -- run ibv_pingpong -h to
> see a list of the switches you can try.
>
> The current code is stable for me, but all that means is that my tiny
> selection of tests and test systems has not uncovered any of the bugs
> that are undoubtedly present. Some of the limitations I know about:
>
> - Only RC is implemented. There are not even any functions to call
> to create UD address handles yet.
> - Only Tavor mode is supported -- PCI Express HCAs will not work if
> they are running mem-free firmware.
> - On x86, only CPUs with SSE will work now. I'd be surprised if
> anyone has x86 system with an HCA that doesn't have SSE.
>
> Also, I've only tried 32-bit i386 userspace running on i386 and x86_64
> kernels -- I don't expect any portability problems but I haven't even
> built for other architectures.
>
> In any case, please give this a spin and let me know how it looks to you.
>
> My short- and medium-term plans are:
>
> 1. Catch up on reviewing and applying the patche queue I'm sitting on.
> 2. Land the Arbel mem-free mode support from the roland-uverbs branch
> onto the main trunk (and merge it upstream once 2.6.11 is out and
> 2.6.12 opens).
> 3. Implement UD support for userspace. I should have this done before
> the end of next week.
> 4. Implement mem-free support for userspace.
>
> Thanks,
> Roland
One problem:
infiniband/driver.h includes infiniband/kern-abi.h, which is not
installed.
I worked around that, for now, by copyng it by hand.
--
MST - Michael S. Tsirkin
More information about the general
mailing list