[openib-general] comments on DAT registry in OpenIB

[Bob Woodruff]

Roland wrote,
>I disagree.  It doesn't make sense to me for us to add an abstraction
>layer on top of another abstraction layer -- let's just fix the first
>abstraction layer.

>If we follow the approach of changing the name of DAT to RDMA and then
>putting it in the kernel, we end up with a stack that looks like:

>    upper layer protocol <-> RDMA midlayer <-> IB RDMA provider <-> IB
midlayer <-> >IB low-level driver

>Let's just evolve the IB midlayer so the picture can be more sensible:

>    upper layer protocol <-> RDMA midlayer <-> IB low-level driver

> - R.

At some layer the interface is transport dependent. I think that this
is at the IB core mid-layer. For example, the IB CM and iWarp CM are totally
different. The IB layer has SA queries that are not needed by iWarp.
I think it could look like,

ULP <-> RDMA-midlayer <-> ib-midlayer  <-> H/W driver
                      <-> iwarp-midlayer <-> H/W driver

where 
       ib midlayer and iWarp midlayer can share common code for functions
       that both need. For example, memory pinning and such might be common
       so you could have
      
       ib_reg_mem() {
          common_memory_pin()
          register_mem_with_IB_HCA()
        }

        iWarp_reg_mem()
           common_memory_pin()
           register_mem_with_iWarp_Nic()
        }

Also starting with separate mid-layers for ib and iwarp allows them to 
initially be developed independently and in parallel, with the factoring out
of common code later and not totally disrupting the ib tree.

my 2 cents.

woody