[ofw] Why is __srp_pnp_cb called when the subnet is up?

Tue Jul 3 01:08:40 PDT 2007

Fab Tillier wrote:

> It's not that the iSCSI target didn't respond to DM queries, but rather
> that the device ID generated (based on the information in the I/O
> Controller profile using the rules in the IB spec for creating HW IDs)
> didn't match the SRP driver.  Standard driver matching rules apply for
> SRP.
>
> I don't know how iSER over IB works - do the targets get discovered like
> regular I/O Units and I/O Controllers, or is it done via IPoIB and some
> other iSCSI discovery mechanism?
>   
Targets may be discovered in several ways:

    * Manually: by connecting to a specific machine (over TCP) and
      sending a login request. Of course, the connection request may be
      rejected (if the other machine doesn't listen on that port), or it
      can reject the login request (by sending a login response with an
      error code).
    * iSNS: the initiator gets a list of targets from an iSNS server.
      iSCSI targets with iSNS support register themselves in the iSNS
      server.
    * SLP: the initiator sends a (broadcast) SLP query (saying that it
      is looking for iSCSI targets). iSCSI targets with SLP support
      replay to that query.

Unlike the SRP initiator that is loaded only if SRP targets exist in the
fabric, an iSCSI initiator should be able to start even if no iSCSI
target exists (this is what happens with Microsoft's iSCSI initiator). I
wonder - do you think that I need to use ibiou for iSER (or is ibiou
good only for the SRP model)?

> If the target doesn't respond to DM queries you wouldn't see it get
> reported in the device manager.  If it is getting reported, but no
> driver is loading, it's a device ID issue.  If it's not getting reported
> at all, it's a DM query issue.
>
> IBAL doesn't provide path PnP notifications for devices other than IOU
> and IOCs (it might even be limited to IOC only).  You can however issue
> path queries from your driver.  This assumes that there is some
> discovery mechanism that allows you to identify the target port GID so
> that you can issue such a query.
>   
Yes. I plan to do that like what we do in Linux with the RDMA CM (i.e.
using arp-send over IPoIB to retrieve the remote address info).

Thanks,
Erez
> Hopefully that helps!
> -Fab
>
> -----Original Message-----
> From: Erez Zilber [mailto:erezz at voltaire.com] 
> Sent: Friday, June 29, 2007 3:21 AM
> To: Fab Tillier; ofw at lists.openfabrics.org
> Subject: RE: [ofw] Why is __srp_pnp_cb called when the subnet is up?
>
> Fab,
>  
> Thanks for the quick answer. Before I ask more questions, I will explain
> the motivation for my questions: we are working on an iSCSI over iSER
> driver for windows. I'm trying to use the SRP driver as a reference for
> some parts of my driver.
>  
> I've noticed that when I have a SRP target on the fabric, __srp_pnp_cb
> is called and the SRP initiator tries to connect. This doesn't happen
> when I have an iSCSI over iSER target (instead of the SRP target). Is it
> correct to say that __srp_pnp_cb isn't called because the SRP target
> didn't respond to DM queries?
>  
> You said that "when a path is added it attempts to connect to the
> target". I need to be able to connect to a target (even if it doesn't
> respond to DM queries) when a path is discovered. How do you suggest to
> do that?
>  
> Thanks,
> Erez
>
> ________________________________
>
> From: Fab Tillier [mailto:ftillier at windows.microsoft.com]
> Sent: Thu 6/28/2007 10:33 PM
> To: Erez Zilber; ofw at lists.openfabrics.org
> Subject: RE: [ofw] Why is __srp_pnp_cb called when the subnet is up?
>
>
>
> Hi Erez,
>
> When the port goes active (due to the SM configuring it) the bus driver
> sweeps the fabric looking for I/O Units.  When it finds the I/O Units it
> creates a device object for it, and loads ibiou.sys as the function
> driver.  Ibiou.sys creates device objects for all child I/O controllers
> of the I/O Unit, in your case SRP Controllers.  The SRP driver is then
> loaded as the function driver for those device nodes.  The SRP driver
> registers for IB PnP events for path notifications, and when a path is
> added it attempts to connect to the target.
>
> The target doesn't need to do much aside from responding to device
> management queries.  If it didn't respond to DM queries no driver would
> get loaded.
>
> Hopefully that helps.
>
> -Fab
>
> -----Original Message-----
> From: ofw-bounces at lists.openfabrics.org
> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of Erez Zilber
> Sent: Thursday, June 28, 2007 7:49 AM
> To: ofw at lists.openfabrics.org
> Subject: [ofw] Why is __srp_pnp_cb called when the subnet is up?
>
> Hi,
>
>
> I'm going over the srp code, and I saw that when I start opensm, the srp
> driver is loaded, and I see the following call stack:
>
>
> srp!__srp_connect_sessions
>
> srp!__srp_connect_path
>
> srp!__srp_pnp_cb
>
> ibbus!__notify_users
>
> ibbus!__report_path
>
> ibbus!__add_ioc_paths
>
> ibbus!__report_ioc_add
>
> ibbus!__add_iocs
>
> ibbus!__report_iou_add
>
> ibbus!__add_ious
>
> ibbus!ioc_pnp_process_reg
>
> ibbus!__cl_async_proc_worker
>
> ibbus!__cl_thread_pool_routime
>
> ibbus!__thread_callback
>
> nt!PspSystemThreadStartup
>
> nt!KxStartSystemThread
>
>
> My questions is: what is the trigger for these events? Is it because the
> subnet is up or because the srp target on the other side did something
> when its IB interface became active?
>
>
> Thanks,
>
> --
>
> ____________________________________________________________
>
> Erez Zilber | 972-9-971-7689
>
> Software Engineer, Storage Team
>
> Voltaire - _The Grid Backbone_
>
> __
>
> www.voltaire.com <http://www.voltaire.com/>
>
>
>
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>
>
>
>   

<http://www.voltaire.com/>