[openib-general] Re: Continue to experience problems in installing Gen2 on IA-32

[Roland Dreier]

    Roland> Do you have any theory as to why the drivers worked in
    Roland> 64-bit mode and failed in 32-bit mode?  I don't see any
    Roland> reason why the parameters passed to INIT_IB would be any
    Roland> different.

    Grant> grundler at gsyprf3:/usr/src/openib_gen2/src/linux-kernel/infiniband/hw/mthca$ fgrep writeq *
    Grant> ...
    Grant> mthca_doorbell.h:       __raw_writeq((__force u64) val, dest);
    Grant> mthca_doorbell.h:       __raw_writeq(*(u64 *) val, dest);

It's a good theory, except for two problems:
 - that code is only used for data-path doorbell ringing, so it won't
   be used until long after the INIT_IB command.
 - It's inside an "#if BITS_PER_LONG" block, so it doesn't ever get
   used on 32-bit platforms anyway.

    Grant> The only theory I can think of is 64-bit MMIO writes on a
    Grant> 32-bit OS will come out as two seperate writes. But since
    Grant> others are using this without a problem, this isn't likely
    Grant> a generic issue. Maybe there is some timing issue here...ie
    Grant> slower/faster CPU or chipset is exposing a problem.

Well, earlier in the thread it was said that the same system worked
fine with a 64-bit kernel and failed with a 32-bit kernel.  So it's
probably not a chipset issue.

And the INIT_IB command isn't using MMIO writes to transfer the
command block to the HCA -- the HCA DMAs it out of memory itself.

 - R.