[openib-general] [PATCH v3 0/6] Tranport Neutral Verbs Proposal.

[Krishna Kumar]

[REPOSTING - as 2 patches in my first set didn't seem to have got to openib
mailing list though I checked after 2 hours. Apologies in advance since people
will get duplicates of atleast 4 of the successful patches]

This patchset is a proposal to create new API's and data structures with
transport neutral names. The idea is to remove the old API once all
libraries/applications/examples are gradually converted to use the new API.

Patch 1/6 - Changes to libibverbs configuration file to build the libibverbs
	    with the new API.
Patch 2/6 - Additions to include files in libibverbs for the new API.
Patch 3/6 - Source files in libibverbs defining the new API.
Patch 4/6 - Convert librdmacm examples to use the new API.
Patch 5/6 - Convert librdmacm include files to use the new libibverbs API.
Patch 6/6 - Convert librdmacm source files to use the new libibverbs API.

Changes from previous (v2) round :
----------------------------------

1. #defined most data structures as suggested by Sean. Created a deprecate.h
   which can be removed once all apps are converted to new API.

2. Changed rdma_ to rdmav_. This also enabled to retain rdma_create_qp() and
   rdma_destroy_qp() routines.

(The last suggestion to not convert IB specific types to generic types was not
 done at this time, since my previous note explained that it is not clear how to
 retain some names while changing others. Eg, ibv_event_type or
 ibv_qp_attr_mask, which have enums for both IB and generic types, etc).


Testing :
---------

1. Compile tested libibverbs, librdmacm, libmthca, libsdp, libibcm,
   libipathverbs and dapl. No warnings or failures in any of these. The
   only warning in libmthca was regarding multiple ibv_read_sysfs_file()'s,
   which is not a compile issue (and also can be removed).

2. Tested rping, ibv_devices & ibv_devinfo utils.


Information notes found during the changes :
--------------------------------------------

1. Added LIBRDMAVERBS_DRIVER_PATH and also use old OPENIB_DRIVER_PATH_ENV for
   backwards compatibility, but have not set user_path to include
   OPENIB_DRIVER_PATH_ENV results.

2. Currently ibv_driver_init is implemented in all drivers. But the function
   returns a "struct ibv_driver *", while we expect "struct rdma_driver *". In
   reality this is fine as they are both pointers pointing to identical objects.
   Otherwise each driver has to be changed now. Once all drivers are changed to
   use rdma_* API's, this will not be an issue.

3. All names are changed to neutral names, even IB specific names as it is not
   clear how to retain some names while changing others. Eg, ibv_event_type
   or ibv_qp_attr_mask, etc.

4. Passing different pointer to verbs, though the end result is the same (no
   warnings generated though as this is a link-time trick). Eg :

	int rdma_query_device(struct rdma_context *context,
			      struct rdma_device_attr *device_attr)
	{
		return context->ops.query_device(context, device_attr);
	}
   However this will not be an issue once the drivers are changed to use the
   new API. Eg : 
  	int mthca_query_device(struct rdma_context *context,
			       struct rdma_device_attr *attr)

5. Makefile.am still makes libibverbs.* libraries so that other apps do not
   break. librdmaverbs.spec.in also does the same.

6. Kept ibv_driver_init call as all libraries have implemented ibv_driver_init,
   but this can be changed easily to new API (and then retired).

7. Prefix is kept as rdmav_ (rdv_ didn't have much takers) to be generic and
   consistent enough.

8. [Missing] IBV_OPCODE() macro is not done (but no use ones it currently).

---

Signed-off-by: Krishna Kumar <krkumar2 at in.ibm.com>