[openib-general] Re: SPAM: [PATCH] [RFC] - dapl - dat_ep_free() can return without freeing the endpoint

[Steve Wise]

On Tue, 2006-04-04 at 15:05 -0700, Sean Hefty wrote:
> James Lentini wrote:
> >> void dapli_destroy_conn(struct dapl_cm_id *conn)
> >> {
> >> 	int in_callback;
> >>+	struct rdma_cm_id *cm_id;
> >> 
> >> 	dapl_dbg_log(DAPL_DBG_TYPE_CM, 
> >> 		     " destroy_conn: conn %p id %d\n",
> >> 		     conn,conn->cm_id);
> >>-
> >> 	dapl_os_lock(&conn->lock);
> >> 	conn->destroy = 1;
> >> 	in_callback = conn->in_callback;
> >>-	dapl_os_unlock(&conn->lock);
> >>-
> >>-	if (!in_callback) {
> >>-		if (conn->ep)
> >>-			conn->ep->cm_handle = IB_INVALID_HANDLE;
> >>-		if (conn->cm_id) {
> >>-			if (conn->cm_id->qp)
> >>-				rdma_destroy_qp(conn->cm_id);
> >>-			rdma_destroy_id(conn->cm_id);
> >>+	do {
> >>+		if (in_callback) {
> >>+			dapl_os_unlock(&conn->lock);
> >>+			usleep(10);
> >>+			dapl_os_lock(&conn->lock);
> 
> In general this doesn't work.  The calling thread may be the callback thread, 
> which would lead to deadlock.  This is why we don't just call rdma_destroy_id() 
> directly, and let it wait for the callback to complete.

I'm not sure we can get a deadlock here.  This is in user space.  And
the "callback thread" is a thread created by the dapl library to process
ibv and cma events.  It just posts events to a software EVD and maybe
wakes up a consumer thread.  Its not like the kernel-mode direct
function calls. 

Looking through the code I don't see where dapli_destroy_conn() can be
called by the function that sets in_callback=1.