[openib-general] Re: [PATCH] [CMA] support loopback addresses

[Sean Hefty]

Michael S. Tsirkin wrote:
>>To keep rdma_resolve_addr() generic, it is an asynchronous call.  The work
>>queue is used to callback the user from a separate thread other than the one
>>that they called down with.  The ib_addr module does something similar when
>>the destination address is actually a local address, deferring the callback to
>>another thread.
> 
> Maybe this should be moved to ib_addr then?
> 
>>The alternative is to have the API behave one way for destination addresses that 
>>are local, versus those that are remote, but this complicates applications that 
>>are not aware if an address belongs to the local or a remote system.
> 
> So, maybe we can reuse the ib_addr wq? Having all requests complete from
> a single thread could help applications, I think.

I did consider pushing this down into ib_addr because it has an existing work 
queue, and it would have been easier to implement, but ib_addr simply converts 
an IP address into a GID/pkey.  To keep the CMA RDMA centric, rather than 
IB-specific, it seemed better to me to map lookback addresses to an RDMA device 
outside of IB specific code.  For example, if the mapping were moved into 
ib_addr, then loopback addresses would not work for a system containing only 
iWarp devices.

If there's concern about using too many work queues in the IB stack, we could 
export a single work queue from ib_core that's accessible to multiple modules. 
Work queues appear to be used by these modules: ib_mad, rdma_cm, ib_cm, ib_addr, 
  sdp, and ipoib.

- Sean