[openib-general] Re: RDMA Generic Connection Management

[James Lentini]

On Tue, 30 Aug 2005, Roland Dreier wrote:

> I just committed this SRP fix, which should make sure we don't use a
> device after it's gone.  And it actually simplifies the code a teeny bit...

The device could still be used after it's gone. For example:

 - the user is configuring SRP via sysfs. The thread in 
   srp_create_target() has just called ib_sa_path_rec_get() 
   [srp.c line 1209] and is waiting for the path 
   record query to complete in wait_for_completion()
 - the SA callback, srp_path_rec_completion(), is called. This 
   callback thread will make several verb calls (ib_create_cq, 
   ib_req_notify_cq, ib_create_qp, ...) without any coordination with 
   the hotplug device removal callback, srp_remove_one 

Notice that if the SA client's hotplug removal function, 
ib_sa_remove_one(), ensured that all callbacks had completed before 
returning the problem would be fixed. This would protect all ULPs from 
having to deal with hotplug races in their SA callback function. The 
fix belongs in the SA client (the core stack), not in SRP.

All the ULPs are deficient with respect to their hotplug 
synchronization. Given that there is a common problem, doesn't it make 
sense to try and solve it in a generic way instead of in each ULP?