[openib-general] Re: Continue to experience problems in installing Gen2 on IA-32

[Weikuan Yu]

Hi,

Thanks for your suggestions. Your guess is right with the debugging 
statement. Output is at the end.

Weikuan

--------------------------------------
ib_mthca: Mellanox InfiniBand HCA driver v0.06 (June 23, 2005)
ib_mthca: Initializing  (0000:03:00.0)
ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 24 (level, low) -> IRQ 177
ib_mthca 0000:03:00.0: Found bridge:  (0000:02:03.0)
ib_mthca 0000:03:00.0: FW version 000300030002, max commands 64
ib_mthca 0000:03:00.0: FW size 6143 KB (start e7a00000, end e7ffffff)
ib_mthca 0000:03:00.0: HCA memory size 131071 KB (start e0000000, end 
e7ffffff)
ib_mthca 0000:03:00.0: Max port width: 2
ib_mthca 0000:03:00.0: Max QPs: 16777216, reserved QPs: 1024, entry 
size: 256
ib_mthca 0000:03:00.0: Max SRQs: 1024, reserved SRQs: 16, entry size: 32
ib_mthca 0000:03:00.0: Max CQs: 16777216, reserved CQs: 128, entry 
size: 64
ib_mthca 0000:03:00.0: Max EQs: 64, reserved EQs: 1, entry size: 64
ib_mthca 0000:03:00.0: reserved MPTs: 16, reserved MTTs: 16
ib_mthca 0000:03:00.0: Max PDs: 16777216, reserved PDs: 0, reserved 
UARs: 1
ib_mthca 0000:03:00.0: Max QP/MCG: 16777216, reserved MGMs: 0
ib_mthca 0000:03:00.0: Flags: 00370347
ib_mthca 0000:03:00.0: profile[ 0]--10/20 @ 0x        e0000000 (size 0x 
4000000)
ib_mthca 0000:03:00.0: profile[ 1]-- 0/16 @ 0x        e4000000 (size 0x 
1000000)
ib_mthca 0000:03:00.0: profile[ 2]-- 7/18 @ 0x        e5000000 (size 0x 
  800000)
ib_mthca 0000:03:00.0: profile[ 3]-- 9/17 @ 0x        e5800000 (size 0x 
  800000)
ib_mthca 0000:03:00.0: profile[ 4]-- 3/16 @ 0x        e6000000 (size 0x 
  400000)
ib_mthca 0000:03:00.0: profile[ 5]-- 4/16 @ 0x        e6400000 (size 0x 
  200000)
ib_mthca 0000:03:00.0: profile[ 6]--12/15 @ 0x        e6600000 (size 0x 
  100000)
ib_mthca 0000:03:00.0: profile[ 7]-- 8/13 @ 0x        e6700000 (size 0x 
   80000)
ib_mthca 0000:03:00.0: profile[ 8]--11/11 @ 0x        e6780000 (size 0x 
   10000)
ib_mthca 0000:03:00.0: profile[ 9]-- 2/10 @ 0x        e6790000 (size 0x 
    8000)
ib_mthca 0000:03:00.0: profile[10]-- 6/ 5 @ 0x        e6798000 (size 0x 
     800)
ib_mthca 0000:03:00.0: HCA memory: allocated 106082 KB/124928 KB (18846 
KB free)
b_mthca 0000:03:00.0: Allocated EQ 1 with 65536 entries
ib_mthca 0000:03:00.0: Allocated EQ 2 with 128 entries
ib_mthca 0000:03:00.0: Allocated EQ 3 with 128 entries
ib_mthca 0000:03:00.0: Setting mask 00000000000f43fe for eqn 2
ib_mthca 0000:03:00.0: Setting mask 0000000000000400 for eqn 3
ib_mthca 0000:03:00.0: NOP command IRQ test passed
ib_mthca 0000:03:00.0: Command 09 completed with status 03
ib_mthca 0000:03:00.0: INIT_IB returned status 03.
ib_mthca 0000:03:00.0: Command 09 completed with status 03
ib_mthca 0000:03:00.0: INIT_IB returned status 03.



On Aug 11, 2005, at 6:30 PM, Roland Dreier wrote:

>     Weikuan> At the end of this email, I have included the output from
>     Weikuan> our system when enabling
>     Weikuan> CONFIG_INFINIBAND_MTHCA_DEBUG=y. Note that there are
>     Weikuan> additional four lines of warning message during the
>     Weikuan> initiation of the device. These are generated from
>     Weikuan> init_port() function, due to the incorrect return status
>     Weikuan> of a command to the firmware, INIT_IB.
>
> Did these warning messages about INIT_IB not show up in the kernel
> before you enabled CONFIG_INFINIBAND_MTHCA_DEBUG?  They are printed
> using mthca_warn(), which should be printed no matter what.
>
> In any case I guess you built your firmware image without support for
> 1X.  Is this right?
>
> Do you have any theory as to why the drivers worked in 64-bit mode and
> failed in 32-bit mode?  I don't see any reason why the parameters
> passed to INIT_IB would be any different.
>
> Anyway, can you apply the debugging patch below and send the output
> you get during device initialization (with 
> CONFIG_INFINIBAND_MTHCA_DEBUG
> enabled, of course)?  I'm guessing you'll see something like:
>
>     ib_mthca 0000:02:00.0: Max port width: 2
>
> If my guess is correct, then we can use that value to get the correct
> width to pass back to INIT_IB.
>
> Thanks,
>   Roland
>
> Index: infiniband/hw/mthca/mthca_cmd.c
> ===================================================================
> --- infiniband/hw/mthca/mthca_cmd.c	(revision 3056)
> +++ infiniband/hw/mthca/mthca_cmd.c	(working copy)
> @@ -1031,6 +1031,7 @@ int mthca_QUERY_DEV_LIM(struct mthca_dev
>  	MTHCA_GET(size, outbox, QUERY_DEV_LIM_UAR_ENTRY_SZ_OFFSET);
>  	dev_lim->uar_scratch_entry_sz = size;
>
> +	mthca_dbg(dev, "Max port width: %x\n", dev_lim->max_port_width);
>  	mthca_dbg(dev, "Max QPs: %d, reserved QPs: %d, entry size: %d\n",
>  		  dev_lim->max_qps, dev_lim->reserved_qps, dev_lim->qpc_entry_sz);
>  	mthca_dbg(dev, "Max SRQs: %d, reserved SRQs: %d, entry size: %d\n",
>