[openib-general] ISER cleanup

[Grant Grundler]

On Tue, Aug 23, 2005 at 07:11:18PM +0300, Dan Bar Dov wrote:
> Some of the comments were taken care of in today's commits
> iovecs removed
> procfs removed
> Function entry/exit traces removed
> Unnecessary files removed:
> kernel_dep.h
> iser_bhs.h
> iser_trace.c

ok - thanks

> All DAT 1.2 comments removed.
> Actually the current code is not DAT 1.2 compatible,
> but the openIB flavor compatible.

Certianly. It would be good if the code contained a reference
to it's origin. It's obvious to those who read this email thread
or folks looking at svn.openib.org. But it won't be once the
code gets into

> Since work started on a CM abstraction, I expect ISER to get
> off of kdapl and onto ib-verbs + CM abstraction.

Understood. I lobbied heavily yesterday at the conference to remove
elements from kapld with similar functionality to openib verbs
and create a "rdma_cm" module with the leftovers.

> > 	I was expecting iSCSI framework to define the data structures
> > 	it needs to talk to a service provider.
> This is an "extended API". The ISER spec defines an ISER API, but it does
> not consider implementation. We chose to implement the extra api out of
> the iser_api structure and in the iser_ext_api struct. 

ok

> iSCSI is still not part of the kernel so we had first modified and
> added the datamover framework to linux-iscsi and now to open-iscsi.
> Once open-iscsi is in the kernel we'll use it as the framework.

ok - I'm not following that developement and trust some of the
other folks who are.

...
> We do not understand some of the substance issues - for example,
> dma related comments - are taken care of by iscsi, not the transport.
> The io_mmu comment, we completely do not understand - there was some
> platform specific code, but its all gone now.

What Christoph said. ULPs are responsible for mapping the buffer virtual
address to a "DMA address".  The right kernel interface depends on
the kernel version:
	2.2	virt_to_phys, phys_to_virt
			no IOMMU support.

	2.4	pci_map_single, pci_map_sg, pci_unmap_*, et al.
			See Documentation/DMA-mapping.txt.
			Only defined to support "PCI-like" busses/devices.

	2.6	dma_map_single, dma_map_sg, et al.
			See Documentation/DMA-API.txt
			Supports IOMMU, multiple bus types, and
			non-coherent IO (ie CPU caches not coherent w/DMA).


amd64, IA64, PPC, parisc, alpha, and sparc all have HW with IOMMU.
You need to use 2.6 DMA support in iSER driver.

See the following for examples of correct usage:
.../openib_gen2/src/linux-kernel/infiniband/ulp$ fgrep -l -e dma_map -e dma_unmap */*c
ipoib/ipoib_ib.c
sdp/sdp_buff.c
sdp/sdp_rcvd.c
sdp/sdp_recv.c
sdp/sdp_send.c
sdp/sdp_sent.c
srp/ib_srp.c



> BTW the code is now down to 7K LOC + 2K LOC of heavily commented header files.

getting there...good work!

hth,
grant