[openib-general] [PATCH 5/6] [RFC] iser handling of memory for RDMA

[Or Gerlitz]

Christoph Hellwig wrote:
>  > use kmap_atomic instead of page_address in the code copying from/to SG
>  > which is unaligned for rdma
> 
> this isn't entirely correct I think.  iser_finalize_rdma_unaligned_sg
> is called from a tasklist, which is softirq context, so you can't use
> KM_USER0 there. KM_SOFTIRQ0 should probably work.  

This is exactly the case, iser_finalize_rdma_unaligned_sg runs in 
tasklet context and its code is using KM_SOFTIRQ0 and 
iser_start_rdma_unaligned_sg runs in kernel thread or user process 
context and its code uses KM_USER0

 > Otoh tasklets are not very scalable because tasklets of a type a
 > serialized against running at multiple cpus, so maybe you should
 > switch to a different mechanisms.

I see.

Well, first, the current iser code is not instrumented for completion 
reaping from the CQ to run from multiple contexts. Second, the code in 
iscsi_iser.c borrowed from drivers/scsi/iscsi_tcp.c assumes it runs in 
tasklet context (ie spin_lock_bh() and friends) as the tcp upcalls are.

So for now we plan to keep using tasklets and it does not seem an issue
for the upstream inclusion. Later down the road (specifically for open 
source iser target...) i guess an implementation of multiple (num cpus) 
kernel threads competing on polling the CQ would be considered.

Does this makes sense?

Or.