[openib-general] Re: ANNOUNCE: First usable version of userspace verbs

[Michael S. Tsirkin]

Quoting r. Michael S. Tsirkin <mst at mellanox.co.il>:
> Subject: Re: ANNOUNCE: First usable version of userspace verbs
> 
> Quoting r. Roland Dreier <roland at topspin.com>:
> > Subject: ANNOUNCE: First usable version of userspace verbs
> > 
> > I'm happy to announce the initial availability of userspace verbs
> > support for brave testers.
> > 
> > To try this out, check out the roland-uverbs subversion branch:
> > 
> >     svn co https://openib.org/svn/gen2/branches/roland-uverbs
> > 
> > and build as usual.  Select CONFIG_INFINIBAND_USER_VERBS to build
> > userspace verbs support.
> > 
> > If you want to use a linux-2.6.10 kernel, you will need to apply the
> > new linux-2.6.10-backports.diff patch from the branch (which just
> > exports get_sb_pseudo()).  No patches at all are required for an
> > up-to-date BK or linux-2.6.11-rc4 tree.
> > 
> > If you use udev, add the rule
> > 
> >     KERNEL="uverbs*", NAME="infiniband/%k", MODE="0666"
> > 
> > to your configuration.  Otherwise, create the required device files:
> > 
> >     mknod /dev/infiniband/uverbs0 c 231 128
> >     mknod /dev/infiniband/uverbs1 c 231 129
> > 
> > and so on for as many HCAs as you have installed.
> > 
> > The build the userspace libraries in src/userspace/libibverbs and
> > src/userspace/libmthca with the usual
> > 
> >     ./autogen.sh && ./configure && make && sudo make install
> > 
> > passing whatever parameters to configure you want; you can use
> > --prefix to install to another location.  If you set a non-standard
> > prefix, it may be useful to pass a -I<path> in CPPFLAGS to the
> > configure for libmthca.
> > 
> > Once you have the libraries built and installed, load the ib_mthca and
> > ib_uverbs modules. By default, libibverbs will search for driver
> > libraries in <prefix>/lib/infiniband; if you installed libmthca
> > somewhere else, set the OPENIB_DRIVER_PATH environment variable to
> > point to the directory with mthca.so.
> > 
> > To actually try things out, you can use the ibv_pingpong program
> > shipped as part of the libibverbs package.  For example, one one
> > system start the server side
> > 
> >     $ ibv_pingpong
> > 
> > and on another system start the client by passing the address of the
> > server (in this example I use IPv6 over IPoIB):
> > 
> >     $ ibv_pingpong fe80::202:c901:7fc:c711%ib0
> > 
> > The pingpong program has a number of options -- run ibv_pingpong -h to
> > see a list of the switches you can try.
> > 
> > The current code is stable for me, but all that means is that my tiny
> > selection of tests and test systems has not uncovered any of the bugs
> > that are undoubtedly present.  Some of the limitations I know about:
> > 
> >  - Only RC is implemented.  There are not even any functions to call
> >    to create UD address handles yet.
> >  - Only Tavor mode is supported -- PCI Express HCAs will not work if
> >    they are running mem-free firmware.
> >  - On x86, only CPUs with SSE will work now.  I'd be surprised if
> >    anyone has x86 system with an HCA that doesn't have SSE.
> > 
> > Also, I've only tried 32-bit i386 userspace running on i386 and x86_64
> > kernels -- I don't expect any portability problems but I haven't even
> > built for other architectures.
> > 
> > In any case, please give this a spin and let me know how it looks to you.
> > 
> > My short- and medium-term plans are:
> > 
> >   1. Catch up on reviewing and applying the patche queue I'm sitting on.
> >   2. Land the Arbel mem-free mode support from the roland-uverbs branch
> >      onto the main trunk (and merge it upstream once 2.6.11 is out and
> >      2.6.12 opens).
> >   3. Implement UD support for userspace.  I should have this done before
> >      the end of next week.
> >   4. Implement mem-free support for userspace.
> > 
> > Thanks,
> >   Roland
> 
> One problem:
> infiniband/driver.h includes infiniband/kern-abi.h, which is not
> installed.
> 
> I worked around that, for now, by copyng it by hand.
> 
> -- 
> MST - Michael S. Tsirkin
> 

Neither is infiniband/opcode.h

-- 
MST - Michael S. Tsirkin