[ofa-general] RE: Limited number of multicasts groups that can be joined?

[Sean Hefty]

>I've run into a problem where it appears that I cannot join more than 14
>multicast groups from a single HCA.  I'm using the RDMA CM UD/multicast
>interface from an OFED v1.2 nightly build, and using a '0' address when
>joining to have the SM allocate an unused address.  The first 14
>rdma_join_multicast() calls succeed, a MULTICAST_JOIN event comes
>through for each of them and everything works.  But the 15th call to
>rdma_join_multicast() returns -1 and sets errno to 99, 'Cannot assign
>requested address'.

I was able to join a total of 6 times before I started seeing failures.  Each
join is done by a separate process with their own QP.  I'll track down the
failure in more detail, but it will likely take me a couple of days to look into
this.

>This feels like a bug to me; though regardless this limit is WAY too
>low.  Any ideas what might be going on, or how I can work around it?

At least on my systems, I see device attributes of:

max_mcast_grp = 8192
max_mcast_qp_attach = 8
max_total_mcast_qp_attach = 65536

- Sean