[Openib-windows] Fabric usage at boot time

[Jan Bottorff]

A GREAT start would be to make all the fabric drivers needed to access
storage be boot load drivers (or at least know they work correctly when
loaded at boot time). This may be really easy or it may be really hard.
At OS startup time there are some limitations on the environment drivers
run in. There is unfortunately no documentation I know of that tells you
these limitations. It also varies a bit with different OS versions (W2K
vs. W2K3).

When working on drivers that have to work at boot time I generally
alternate between setting them to normal demand load drivers (which are
a bit easier to test and debug), and then periodically setting them to
be boot load, and see what needs fixing (usually by seeing what API's
fail). 

There also are considerations for putting paging files on the fabric.
For example, if your supporting a paging file you can't take any page
faults while reading or writing the paging file. This probably means you
can't take any page faults to reestablish a connection to a boot disk
which has temporarily failed or during normal read/writes. Booting from
the fabric implies paging to the fabric. It also would be desirable to
write crash dumps to the fabric boot disk, which has it's own set of
issues.

Windows 200x also can't boot using PXE. You can install to a local hard
disk using PXE. There MUST be INT 13 block I/O services provided by the
adapter rom to access the boot disk (or EFI bios equivalents on
appropriate systems). NTLDR (loaded by the bios from the system disk)
will want to load the OS kernel, boot.ini, the system hive of the
registry and all drivers marked as boot start. NTLDR talks INT 13 to the
boot disk to read these files. Control is then transferred to the OS
kernel, and these boot loaded drivers MUST start and be capable of
accessing the boot disk without real mode INT 13 help. The TCP/IP stack
is not a boot load driver, so you can't boot using IPoIB (unless you
supply your own TCP/IP stack as part of a storage driver). Just to be
squeeky correct, you could have a single unified storage miniport driver
be used by NTLDR to access the boot disk (this is usually a carefully
written scsi miniport driver copied to the system disk as ntbootdd.sys).
Since the IB hardware drivers+fabric drivers+storage miniport are not a
single driver, this method doesn't work.

- Jan

> -----Original Message-----
> From: Fab Tillier [mailto:ftillier at silverstorm.com] 
> Sent: Friday, October 28, 2005 8:41 PM
> To: Jan Bottorff; openib-windows at openib.org
> Subject: RE: [Openib-windows] Fabric usage at boot time
> 
> > From: Jan Bottorff [mailto:jbottorff at xsigo.com]
> > Sent: Friday, October 28, 2005 7:18 PM
> > 
> > I was noticing none of the the fabric or hw drivers are 
> designated as 
> > boot load drivers in the INF.
> > 
> > Is there any plan to do the work required to make all the 
> drivers work 
> > at boot time, allowing the possibility of creating systems that can 
> > boot over the IB fabric?
> 
> I would love to see boot over IB functionality in the Windows 
> OpenIB stack.  I don't have any hardware to test this, 
> though, so it will likely be delayed until someone else that 
> does have hardware tries it, or I get hardware and time to do 
> so myself.
> 
> There is support for PXE boot over IPoIB, but there isn't any 
> handoff between the BIOS and OS during boot in this configuration.
> 
> I personally don't know enough about the details of the 
> handoff between BIOS environment and the OS to be able to 
> tell what's needed from the stack to support this.  Can you 
> point me to any readings that would help clarify things?
> If there are obvious things that we can do to prepare the 
> drivers for boot support, even in the absence of actual test 
> environment, that would be great.