[ofw] Support boot device in IB stack?

[James Yang]

When OS is doing crash dump, it will only enter a special state where IRQL is 31 and only allow the disk controller driver to run so that the whole memory could be written to crashdump file. If you go through the crashdump process, you will see it. I know it's a bit confusing since MS has no doc on this.

-James

-----Original Message-----
From: Leonid Keller [mailto:leonid at mellanox.co.il] 
Sent: Thursday, February 12, 2009 12:23 AM
To: James Yang; Tzachi Dar; ofw at lists.openfabrics.org
Subject: RE: [ofw] Support boot device in IB stack?

> In order to support crashdump, the IB related code must be able to run
at IRQL 31.
Could you explain why do you think so ?
As far as I know, at IRQL 31 all interrupts/dpc/threads are blocked.
When a crash happens, the only task of OS is to create a dump, while
preserving the current state.
So it looks strange to me, that OS will allow to any driver to continue
working after a crash.
As to creating a crash dump at IRQL 31, it looks a bit strange to me.
OS have to write the dump to HD, which presumes the work with disk
driver at IRQL 2 at worst.
Am I missing something ? 

> -----Original Message-----
> From: ofw-bounces at lists.openfabrics.org 
> [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of James Yang
> Sent: Thursday, February 12, 2009 1:59 AM
> To: Tzachi Dar; ofw at lists.openfabrics.org
> Subject: RE: [ofw] Support boot device in IB stack?
> 
> Hi Tzachi,
> 
> The boot should be OK if #1 and #3 are resolved. In order to 
> support crashdump, the IB related code must be able to run at 
> IRQL 31. It will be good to know what functions are OK to run 
> and what are not. By the way, according to DDK, "Callers of 
> KeAcquireSpinLock must be running at IRQL <= DISPATCH_LEVEL."
> 
> There is no formal document on crashdump. The most important 
> thing is that it's running at IRQL 31, and multiple threading 
> may not work at this time.
> 
> iSCSI is one of the solutions, but we also have to support 
> vhba directly. And in order to get MS WHQL for vhba, we must 
> support crashdump. Do you get any request to support 
> crashdump on iSCSI/ipoib?
> 
> Thanks,
> James 
> 
> -----Original Message-----
> From: Tzachi Dar [mailto:tzachid at mellanox.co.il]
> Sent: Wednesday, February 11, 2009 12:55 AM
> To: James Yang; ofw at lists.openfabrics.org
> Subject: RE: [ofw] Support boot device in IB stack?
> 
> Hi James,
> 
> As for 1,3 this are technical changes that can easily be done.
> 
> Issue #2 seems much more problematic to me.
> First I must say that I haven't studied the topic thoroughly, 
> so I might be getting a wrong impression of the entire issue.
> One more thing to start with, is that I'm not sure that this 
> request is a must. What I mean by that is that booting will 
> take place just fine.
> The one thing that will not work is creating a crash dump file later.
> 
> In any case, spinlocks don't really have a problem with 
> higher IRQL (the way I understand it), but many other 
> commands have. So if the code is only trying to send/receive 
> data with qps that are open I guess that things should work. 
> Trying to open new QPs will not work (we have to wait). 
> Please also note that there are only about 5 commands that 
> can be called from IRQL>DISPATCH_LEVEL.
> 
> Can you please send a reference to the document that says 
> what are the demands after the system has crashed?
> 
> Another question is this, will booting using iSCSI be a 
> better option for you? (not that it was done using ipoib).
> 
> Thanks
> Tzachi
> 
> > -----Original Message-----
> > From: ofw-bounces at lists.openfabrics.org 
> > [mailto:ofw-bounces at lists.openfabrics.org] On Behalf Of James Yang
> > Sent: Tuesday, February 10, 2009 11:31 PM
> > To: ofw at lists.openfabrics.org
> > Subject: [ofw] Support boot device in IB stack?
> > 
> > Hi,
> > 
> > We would like to support boot device especially SAN boot 
> over IB. So 
> > far I can see the following issues in current code:
> > 1) Many functions are page-able. If these functions are running at 
> > boot or shut down time, the disk may not be ready, the paging won't 
> > work.
> > 2) Spinlock may not work for crashdump, whose
> > IRQL>DISPATCH_LEVEL. Any other functions need to be changed
> > when IRQL>DISPATCH_LEVEL?
> > 3) All related drivers should be boot start driver.
> > 
> > Are there any other potential problem? Is there any plan to support 
> > SAN boot?
> > 
> > Please advice.
> > 
> > Thanks,
> > James
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > ofw mailing list
> > ofw at lists.openfabrics.org
> > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
> > 
> 
> _______________________________________________
> ofw mailing list
> ofw at lists.openfabrics.org
> http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ofw
>