[openib-general] Location for iser-target code

[Dan Bar Dov]

On 4/10/06, Mike Christie <michaelc at cs.wisc.edu> wrote:
> Dan Bar Dov wrote:
> > On 4/10/06, Mike Christie <michaelc at cs.wisc.edu> wrote:
> >> Dan Bar Dov wrote:
> >>> On 4/10/06, FUJITA Tomonori <fujita.tomonori at lab.ntt.co.jp> wrote:
> >>>> The OLS abstract may be more informative.
> >>>>
> >>>> http://www.linuxsymposium.org/2006/view_abstract.php?content_key=19
> >>> Is the full document available as well?
> >> The code is shorter than the paper.
> >>
> >>>> In short, tgt is the framework for SCSI target drivers. The
> >>>> combination of tgt and the iSCSI target driver for NIC provides the
> >>>> similar features that IET does.
> >>> SCSI targets are LLDs that sit below the mid-layer. iSCSI target on
> >>> the other hand is a "network" protocol driver, that sits above SCSI
> >>> (sd, st, sg, or directly over mid-layer). Seems like you'd need two
> >>> different frameworks, no?
> >>>
> >> tgt/stgt is probably two frameworks from your point of view. There is a
> >> kernel part for target LLDs to hook into. The kernel part is similar to
> >> scsi-ml, actually it builds onto it and uses some of the scsi-ml
> >> functions, and provides code to share for tasks like creating scatter
> >> lists and mapping commands between the kernel and userspace. The target
> >> LLD basically handles lower level issues like DMAing the data, transport
> >> issues, etc, pretty much what a scsi-ml initiator driver does. For
> >
> > What do you mean by scsi-ml initiator?

Thank you ;-)

>
> SCSI Mid Layer - (this is what the SCSI Layer in linux is referred to
> sometimes)
> initiator - I guess in this context it would be the linux scsi host driver?
>
> So I was referring to drivers like iscsi_tcp, qla2xxx, aic79xx etc.
>

So scsi-ml initiator driver is a scsi LLD. Got it.

> >
> >> iscsi, the tgt lld performs similar tasks as the initiator. It parses
> >> the iscsi PDUs or puts them on the interconnect, handles session and
> >> connection manamgement (this would be done like open-iscsi though), but
> >> then then passes the scsi command to tgt's kernel code.
> >>
> >> The other part of the framework is the userspace component. The tgt
> >> kernel component basically passes scsi commands and task management
> >> functions to a userspace daemon. The daemon contains the scsi state
> >> machine and execute the IO. When it is done it informs the the kernel
> >> component which in turn maps the data into the kernel, forms scatter
> >> lists, and then passes them to the target LLD to send out.
> >>
> > I got completely confused. I understand (obviously wrongly) that to implement an
> > iscsi target (or srp target for that matter), I would write the
> > network facing part in kernel,
> > that would pass the tasks and data to the user mode, the user mode will perform
> > the tasks using scsi drivers (sd/st/sg), and once completed report
> > back to the network facing part.
>
> For tasks did you mean iscsi or scsi type of tasks or is that a rdma or

I meant iscsi/scsi tasks.

> infinniband term too? For software iscsi over tcp we could just do it
> all in usersapce. We are not 100% if a software iscsi target, even it is
> slimmed down and hooks into iscsi_tcp/libiscsi/scsi_transport_iscsi
> would be mergable, because for for software iscsi we could just open a
> socket in userspace to the target and never have to implement new kernel
> code for iscsi or scsi processing. The completely userspace iscsi target
> would listen on the socket for iscsi PDUs, then write/read to the device
> using SG_IO or to some virtualized block device. Arjan gave some good
> review comments about how to do a software iscsi target in userspace
> when IET was posted to linux-scsi for review. He did not hammer out all
> the details though.

I think I start to understand. The tgt framework uses a model similar to
linux's scsi 3 layers. The lld here would be iser, a tcp "data mover",
srp etc. and
they will convert from a wire protocol (iscsi/srp etc.) to scsi, and then pass
the scsi commands to tgt.
The mid-layer equivalent is the tgt, and the upper layer is the command
handler and that is to live in userland, so it can use any block
devices including
md, dm etc.


>
> Could you probably do the same for iser and srp? Maybe we just want a
> userspace framework for these types of software drivers becuase at least
> the read/write to the device part could be shared.
>

I think if I got the model right, it is possible to fit ISER to it.
Again, a lot of the
iser and tcp pdu processing could be shared in a library like you did
for the initiator.

>
> >
> > I guess I'd need a diagram or two to understand :-)
> >
>
> ok, i am not sure if we can post the paper. we will read the OLS rules
> and get back to you.
>

Thanks.
Dan