[ofa-general] Re: How to tell what OFED rev a distro derived IB modules?

[Chris Worley]

On Mon, Apr 27, 2009 at 12:01 PM, Vu Pham <vuhuong at mellanox.com> wrote:
> Vladislav Bolkhovitin wrote:
>>
>> Chris Worley, on 04/27/2009 07:54 PM wrote:
>>>
>>> On Mon, Apr 27, 2009 at 5:03 AM, Bart Van Assche
>>> <bart.vanassche at gmail.com> wrote:
>>>>
>>>> On Mon, Apr 27, 2009 at 8:27 AM, Jack Morgenstein
>>>> <jackm at dev.mellanox.co.il> wrote:
>>>>>
>>>>> The OFED distributions may contain features that the mainstream kernels
>>>>> and libraries do not support.
>>>>> These features frequently require changes in the Infiniband kernel
>>>>> modules.  Such changes are in the form
>>>>> of kernel patches which are applied to the base mainstream kernel on
>>>>> which the OFED release is based.
>>>>> A lag between the mainstream kernel and the OFED kernel is unavoidable,
>>>>> since the new features are first
>>>>> released in the OFED distributions -- and later, gradually (and
>>>>> hopefully), these features make there way
>>>>> into the upstream kernel.
>>>>
>>>> I don't doubt that there is a good reason why new features go in the
>>>> OFED distribution first and later in the mainstream Linux kernel.
>>>
>>> My opinion is: IB is still just too bleeding edge, even for the
>>> vanilla Linux kernel.
>>>
>>> Maybe "Upstream First" is the measure of IB achieving stability.
>>>
>>> SRP (specifically the SCST target code) is my first case in using IB
>>> where I've not been able to start with the latest OFED (or IBGD)
>>> stable release, as OFED is unsupported by the SRP target code, and had
>>> to start with a distro's IB version to get a working SRP target (of
>>> which Ubuntu 8.10 provided the only stable SRP target distro for my
>>> configuration).
>>
>> I think, to find out who's guilty, OFED or SRP target driver, you should
>> simply try the latest SCST/SRP driver from the SCST SVN trunk with the known
>> working OFED. Only make sure you don't have again mixed up older and new
>> SCST headers.
>>
> Here is the simple rule of thumb:
> 1. If you want to use latest/greatest top of trunk SCST SVN, kernel 2.6.28,
> 29... then you have to use the IB driver/modules in that kernel tree. You
> also have to use the ib_srpt driver in SCST SVN tree

Which is what I did, and as RHEL/CentOS5.[23] use newer OFED drivers,
I had to turn to Ubuntu to get a stable SRP target.

>
> 2. If you want to run IB driver, ib_srpt driver and SCST on distribution
> default kernel (RHEL 5,0/1/2 and its family ie. fedora, centos..., sles 10
> sp1/sp2...) then you should use OFED package (with ib_srpt inside the
> package), SCST-1.0.0
> In the OFED-1.xxx/docs directory, there is a readme on how-to ib_srpt

That doesn't work w/ OFED 1.4 and 1.4.1rc3.  OFED's ib_srpt w/ SCST
1.0.0 hangs the sstem during modprobe, similar to what SCST's ib_srpt
did.  The latter was well documented in the SCST list, but later it
was disclosed that SCST doesn't support any incarnation of OFED.

Chris