[openib-general] some issues related to when/why IPoIB calls netif_carrier_on() etc

[Or Gerlitz]

Roland,

I'd like to verify with you an issue related to the logic applied
by ipoib about when/why it decided to join the multicast group used
for IPv4 broadcast (eg ARP) and when/why its sets the carrier bit.

Probing ipoib and ifconfing ib0 to be UP, i see that ib0 is RUNNING,
has an IPv6 address and is joined to the IPv4 broascast group (ie to MGID
ff12:401b:ffff:0:0:0:ffff:ffff), see the exact sequence of operations below.

I wonder if you can shed some light on:

1) what is the exact reason that ib0 is running here, is it as of this
   "magic" configuration of the IPv6 addr that caused it to join to
   the IPv4 and IPv6 broascast groups?

2) is it well defined what conditions should hold s.t IPoIB will be RUNNING

3) just to make sure: RUNNING <--> ipoib called netif_carrier_on(), correct?
   i see that latter is called by ipoib_mcast_join_task(), is it when
   "joining everything we want to join to" holds or you can somehow
   refine the predicate?

thanks,

Or.

$ modprobe ib_ipoib

$ ifconfig ib0 up

$ ifconfig ib0

ib0       Link encap:UNSPEC  HWaddr 00-04-04-04-FE-80-00-00-00-00-00-00-00-00-00-00
          inet6 addr: fe80::208:f104:396:51dd/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
          RX packets:35 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:128
          RX bytes:2760 (2.6 Kb)  TX bytes:456 (456.0 b)

$ ip addr show ib0

10: ib0: <BROADCAST,MULTICAST,UP,10000> mtu 2044 qdisc pfifo_fast qlen 128
    link/[32] 00:04:04:04:fe:80:00:00:00:00:00:00:00:08:f1:04:03:96:51:dd brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
    inet6 fe80::208:f104:396:51dd/64 scope link
       valid_lft forever preferred_lft forever


$ cat /sys/kernel/debug/ipoib/ib0_mcg

GID: ff12:401b:ffff:0:0:0:ffff:ffff
  created: 4294982836
  queuelen:         0
  complete:       yes
  send_only:       no

GID: ff12:601b:ffff:0:0:0:0:1
  created: 4294982838
  queuelen:         0
  complete:       yes
  send_only:       no

GID: ff12:601b:ffff:0:0:0:0:2
  created: 4294983192
  queuelen:         0
  complete:       yes
  send_only:      yes

GID: ff12:601b:ffff:0:0:0:0:16
  created: 4294982840
  queuelen:         0
  complete:       yes
  send_only:      yes

GID: ff12:601b:ffff:0:0:1:ff96:51dd
  created: 4294982838
  queuelen:         0
  complete:       yes
  send_only:       no

$ dmesg

ib0: bringing up interface
ib0: starting multicast thread
ADDRCONF(NETDEV_UP): ib0: link is not ready
ib0: joining MGID ff12:401b:ffff:0000:0000:0000:ffff:ffff
ib0: restarting multicast task
ib0: stopping multicast thread
ib0: waiting for MGID ff12:401b:ffff:0000:0000:0000:ffff:ffff
ib0: join completion for ff12:401b:ffff:0000:0000:0000:ffff:ffff (status -4)
ib0: starting multicast thread
ib0: joining MGID ff12:401b:ffff:0000:0000:0000:ffff:ffff
ib0: join completion for ff12:401b:ffff:0000:0000:0000:ffff:ffff (status 0)
ib0: Created ah ffff810034630380
ib0: MGID ff12:401b:ffff:0000:0000:0000:ffff:ffff AV ffff810034630380, LID 0xc000, SL 0
ib0: successfully joined all multicast groups
ADDRCONF(NETDEV_CHANGE): ib0: link becomes ready
ib0: restarting multicast task
ib0: stopping multicast thread
ib0: waiting for MGID ff12:401b:ffff:0000:0000:0000:ffff:ffff
ib0: adding multicast entry for mgid ff12:601b:ffff:0000:0000:0001:ff96:51dd
ib0: adding multicast entry for mgid ff12:601b:ffff:0000:0000:0000:0000:0001
ib0: starting multicast thread
ib0: joining MGID ff12:601b:ffff:0000:0000:0001:ff96:51dd
ib0: join completion for ff12:601b:ffff:0000:0000:0001:ff96:51dd (status 0)
ib0: Created ah ffff810034630d00
ib0: MGID ff12:601b:ffff:0000:0000:0001:ff96:51dd AV ffff810034630d00, LID 0xc006, SL 0
ib0: joining MGID ff12:601b:ffff:0000:0000:0000:0000:0001
ib0: join completion for ff12:601b:ffff:0000:0000:0000:0000:0001 (status 0)
ib0: Created ah ffff8100346305c0
ib0: MGID ff12:601b:ffff:0000:0000:0000:0000:0001 AV ffff8100346305c0, LID 0xc003, SL 0
ib0: successfully joined all multicast groups
ib0: setting up send only multicast group for ff12:601b:ffff:0000:0000:0000:0000:0016
ib0: no multicast record for ff12:601b:ffff:0000:0000:0000:0000:0016, starting join
ib0: Created ah ffff81003d669400
ib0: MGID ff12:601b:ffff:0000:0000:0000:0000:0016 AV ffff81003d669400, LID 0xc004, SL 0
ib0: setting up send only multicast group for ff12:601b:ffff:0000:0000:0000:0000:0002
ib0: no multicast record for ff12:601b:ffff:0000:0000:0000:0000:0002, starting join
ib0: Created ah ffff81002009f7c0
ib0: MGID ff12:601b:ffff:0000:0000:0000:0000:0002 AV ffff81002009f7c0, LID 0xc005, SL 0
ib0: no IPv6 routers present
ib0: neigh_destructor for ffffff ff12:601b:ffff:0000:0000:0000:0000:0002
ib0: neigh_destructor for ffffff ff12:601b:ffff:0000:0000:0001:ff96:51dd