[Users] kernel: scsi host8: ib_srp: SRP_LOGIN_REJ: requested max_it_iu_len too large
Tom Robinson
tom.robinson at motec.com.au
Thu Jan 16 17:07:59 PST 2020
On Thu, 16 Jan 2020 at 16:44, Bart Van Assche <bvanassche at acm.org> wrote:
>
> On 2020-01-15 18:04, Tom Robinson wrote:
> > I hope this is the right list to ask this question. Please let me know
> > if otherwise.
> >
> > We are trying to configure the following cards on Arch Linux. This is
> > part of a reconfiguration of our infrastructure and change of OS from
> > CentOS to Arch. These cards were working well under CentOS 7 but I can't
> > seem to get them to work under Arch Linux.
> >
> > 05:00.0 InfiniBand: Mellanox Technologies MT25408A0-FCC-QI ConnectX,
> > Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s
> > In... (rev b0)
> > 06:00.0 InfiniBand: Mellanox Technologies MT25408A0-FCC-QI ConnectX,
> > Dual Port 40Gb/s InfiniBand / 10GigE Adapter IC with PCIe 2.0 x8 5.0GT/s
> > In... (rev b0)
> >
> > Currently I am getting the following error on startup of the srp_daemon:
> >
> > kernel: scsi host8: ib_srp: SRP_LOGIN_REJ: requested max_it_iu_len too large
> >
> > Has anyone seen this before? I have done a lot of searching on the
> > internet but as yet have not come across a solution. Can someone please
> > help and shed some light on this?
>
> Which kernel version is running on the Arch Linux system? How have the
> SRP daemon and the ib_srp driver been configured on the initiator
> system? (cat /etc/srp_daemon.conf; cat /etc/modprobe.d/ib_srp.conf)? Is
> the target system a vendor-supplied SRP system or a self-configure Linux
> system? In the latter case, which kernel version has been installed on
> the SRP target system (address fe80:0000:0000:0000:0002:c903:002c:d311)?
> If it is Linux, is the target system using the in-tree (LIO) ib_srpt
> driver or the SCST ib_srpt driver?
>
> Thanks,
>
> Bart.
Hi Bart,
I forgot to mention that on our new Arch config I have IPoIB working.
I can provide that information on request if needed.
My apologies, I also forgot our Subnet Manager in the ACSII Art!
------------ ------------
| lemans | | daytona |
| CentOS 7 | | Arch |
------------ ------------
| | | |
| --------------\ /-------------- |
| X |
| --------------/ \-------------- |
| | | |
| | | |
| | ------------- | |
------------- | atom | -------------
| IB Switch |---|SM(CentOS7)|---| IB Switch |
------------- ------------- -------------
| |
--------------- ----------------
| |
| |
-----------
| OminiOS | "MT25408 ConnectX Mellanox Technologies"
-----------
| |
-----------
| Storage |
-----------
# ibnodes
Ca : 0xf452140300cea310 ports 2 "lemans mlx4_0"
#<----CentOS7
Ca : 0x0002c903002cd2f8 ports 2 "daytona ibp5s0"
#<----ArchLinux
Ca : 0x0002c903002cd30c ports 2 "lemans mlx4_1"
#<----CentOS7
Ca : 0x0002c903002cd310 ports 2 "MT25408 ConnectX Mellanox
Technologies" #<----OmniOS
Ca : 0x0002c903002cd2f4 ports 2 "atom mlx4_0"
#<----CentOS7
Ca : 0xf452140300cea42c ports 2 "daytona ibp6s0"
#<----ArchLinux
Switch : 0x0002c9020048df60 ports 8 "Infiniscale-IV Mellanox
Technologies" base port 0 lid 5 lmc 0
Switch : 0x0002c9020048df58 ports 8 "Infiniscale-IV Mellanox
Technologies" base port 0 lid 2 lmc 0
# ibhosts
Ca : 0xf452140300cea310 ports 2 "lemans mlx4_0"
Ca : 0x0002c903002cd2f8 ports 2 "daytona ibp5s0"
Ca : 0x0002c903002cd30c ports 2 "lemans mlx4_1"
Ca : 0x0002c903002cd310 ports 2 "MT25408 ConnectX Mellanox Technologies"
Ca : 0x0002c903002cd2f4 ports 2 "atom mlx4_0"
Ca : 0xf452140300cea42c ports 2 "daytona ibp6s0"
# ibnetdiscover
#
# Topology file: generated on Fri Jan 17 07:58:01 2020
#
# Initiated from node f452140300cea42c port f452140300cea42e
vendid=0x2c9
devid=0xbd36
sysimgguid=0x2c9020048df63
switchguid=0x2c9020048df60(2c9020048df60)
Switch 8 "S-0002c9020048df60" # "Infiniscale-IV Mellanox Technologies"
base port 0 lid 5 lmc 0
[1] "H-0002c903002cd2f4"[2](2c903002cd2f6) # "atom mlx4_0" lid 6 4xQDR
[3] "H-0002c903002cd310"[2](2c903002cd312) # "MT25408 ConnectX
Mellanox Technologies" lid 8 4xQDR
[5] "H-0002c903002cd2f8"[2](2c903002cd2fa) # "daytona ibp5s0" lid 12 4xQDR
[6] "S-0002c9020048df58"[6] # "Infiniscale-IV Mellanox Technologies" lid 2 4xQDR
[7] "H-f452140300cea310"[2](f452140300cea312) # "lemans mlx4_0" lid 10 4xQDR
[8] "S-0002c9020048df58"[8] # "Infiniscale-IV Mellanox Technologies" lid 2 4xQDR
vendid=0x2c9
devid=0xbd36
sysimgguid=0x2c9020048df5b
switchguid=0x2c9020048df58(2c9020048df58)
Switch 8 "S-0002c9020048df58" # "Infiniscale-IV Mellanox Technologies"
base port 0 lid 2 lmc 0
[1] "H-0002c903002cd2f4"[1](2c903002cd2f5) # "atom mlx4_0" lid 1 4xQDR
[3] "H-0002c903002cd310"[1](2c903002cd311) # "MT25408 ConnectX
Mellanox Technologies" lid 3 4xQDR
[5] "H-f452140300cea42c"[2](f452140300cea42e) # "daytona ibp6s0" lid 11 4xQDR
[6] "S-0002c9020048df60"[6] # "Infiniscale-IV Mellanox Technologies" lid 5 4xQDR
[7] "H-0002c903002cd30c"[2](2c903002cd30e) # "lemans mlx4_1" lid 9 4xQDR
[8] "S-0002c9020048df60"[8] # "Infiniscale-IV Mellanox Technologies" lid 5 4xQDR
vendid=0x2c9
devid=0x673c
sysimgguid=0xf452140300cea313
caguid=0xf452140300cea310
Ca 2 "H-f452140300cea310" # "lemans mlx4_0"
[2](f452140300cea312) "S-0002c9020048df60"[7] # lid 10 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 5 4xQDR
vendid=0x2c9
devid=0x673c
sysimgguid=0x2c903002cd2fb
caguid=0x2c903002cd2f8
Ca 2 "H-0002c903002cd2f8" # "daytona ibp5s0"
[2](2c903002cd2fa) "S-0002c9020048df60"[5] # lid 12 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 5 4xQDR
vendid=0x2c9
devid=0x673c
sysimgguid=0x2c903002cd30f
caguid=0x2c903002cd30c
Ca 2 "H-0002c903002cd30c" # "lemans mlx4_1"
[2](2c903002cd30e) "S-0002c9020048df58"[7] # lid 9 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 2 4xQDR
vendid=0x2c9
devid=0x673c
sysimgguid=0x2c903002cd313
caguid=0x2c903002cd310
Ca 2 "H-0002c903002cd310" # "MT25408 ConnectX Mellanox Technologies"
[1](2c903002cd311) "S-0002c9020048df58"[3] # lid 3 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 2 4xQDR
[2](2c903002cd312) "S-0002c9020048df60"[3] # lid 8 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 5 4xQDR
vendid=0x2c9
devid=0x673c
sysimgguid=0x2c903002cd2f7
caguid=0x2c903002cd2f4
Ca 2 "H-0002c903002cd2f4" # "atom mlx4_0"
[1](2c903002cd2f5) "S-0002c9020048df58"[1] # lid 1 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 2 4xQDR
[2](2c903002cd2f6) "S-0002c9020048df60"[1] # lid 6 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 5 4xQDR
vendid=0x2c9
devid=0x673c
sysimgguid=0xf452140300cea42f
caguid=0xf452140300cea42c
Ca 2 "H-f452140300cea42c" # "daytona ibp6s0"
[2](f452140300cea42e) "S-0002c9020048df58"[5] # lid 11 lmc 0
"Infiniscale-IV Mellanox Technologies" lid 2 4xQDR
Also output from iblinkinfo:
# iblinkinfo
CA: lemans mlx4_0:
0xf452140300cea312 10 2[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 5 7[ ] "Infiniscale-IV Mellanox
Technologies" ( )
CA: daytona ibp5s0:
0x0002c903002cd2fa 12 2[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 5 5[ ] "Infiniscale-IV Mellanox
Technologies" ( )
CA: lemans mlx4_1:
0x0002c903002cd30e 9 2[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 2 7[ ] "Infiniscale-IV Mellanox
Technologies" ( )
Switch: 0x0002c9020048df60 Infiniscale-IV Mellanox Technologies:
5 1[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
6 2[ ] "atom mlx4_0" ( )
5 2[ ] ==( Down/ Polling)==>
[ ] "" ( )
5 3[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
8 2[ ] "MT25408 ConnectX Mellanox Technologies" ( )
5 4[ ] ==( Down/ Polling)==>
[ ] "" ( )
5 5[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
12 2[ ] "daytona ibp5s0" ( )
5 6[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
2 6[ ] "Infiniscale-IV Mellanox Technologies" ( )
5 7[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
10 2[ ] "lemans mlx4_0" ( )
5 8[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
2 8[ ] "Infiniscale-IV Mellanox Technologies" ( )
CA: MT25408 ConnectX Mellanox Technologies:
0x0002c903002cd311 3 1[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 2 3[ ] "Infiniscale-IV Mellanox
Technologies" ( )
0x0002c903002cd312 8 2[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 5 3[ ] "Infiniscale-IV Mellanox
Technologies" ( )
CA: atom mlx4_0:
0x0002c903002cd2f5 1 1[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 2 1[ ] "Infiniscale-IV Mellanox
Technologies" ( )
0x0002c903002cd2f6 6 2[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 5 1[ ] "Infiniscale-IV Mellanox
Technologies" ( )
Switch: 0x0002c9020048df58 Infiniscale-IV Mellanox Technologies:
2 1[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
1 1[ ] "atom mlx4_0" ( )
2 2[ ] ==( Down/ Polling)==>
[ ] "" ( )
2 3[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
3 1[ ] "MT25408 ConnectX Mellanox Technologies" ( )
2 4[ ] ==( Down/ Polling)==>
[ ] "" ( )
2 5[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
11 2[ ] "daytona ibp6s0" ( )
2 6[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
5 6[ ] "Infiniscale-IV Mellanox Technologies" ( )
2 7[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
9 2[ ] "lemans mlx4_1" ( )
2 8[ ] ==( 4X 10.0 Gbps Active/ LinkUp)==>
5 8[ ] "Infiniscale-IV Mellanox Technologies" ( )
CA: daytona ibp6s0:
0xf452140300cea42e 11 2[ ] ==( 4X 10.0 Gbps
Active/ LinkUp)==> 2 5[ ] "Infiniscale-IV Mellanox
Technologies" ( )
More information about the Users
mailing list