[ofa-general] [RFC v2 PATCH 3/5] rdma/cma: add high availability mode attribute to IDs

[Or Gerlitz]

RDMA_ALIGN_WITH_NETDEVICE high availability (ha) mode means that the consumer of
the rdma-cm wants that RDMA sessions would always use the same links (eg <hca/port>)
as the IP stack does.

In the current code, this does not happen when bonding did fail-over but
the link used by an already existing session is operating fine. Consumers
seeking this ha mode would get the new RDMA_CM_EVENT_NETDEV_CHANGE event
when such misalignment happens.

More ha modes can be added in the future.

Signed-off-by: Or Gerlitz <ogerlitz at voltaire.com>

changes from v1 -
- added new event RDMA_CM_EVENT_NETDEV_CHANGE
- took the approach of notifying the user vs disconnecting the ID

Index: linux-2.6.26-rc2/include/rdma/rdma_cm.h
===================================================================
--- linux-2.6.26-rc2.orig/include/rdma/rdma_cm.h	2008-05-15 14:48:44.000000000 +0300
+++ linux-2.6.26-rc2/include/rdma/rdma_cm.h	2008-05-15 14:49:48.000000000 +0300
@@ -53,7 +53,8 @@ enum rdma_cm_event_type {
 	RDMA_CM_EVENT_DISCONNECTED,
 	RDMA_CM_EVENT_DEVICE_REMOVAL,
 	RDMA_CM_EVENT_MULTICAST_JOIN,
-	RDMA_CM_EVENT_MULTICAST_ERROR
+	RDMA_CM_EVENT_MULTICAST_ERROR,
+	RDMA_CM_EVENT_NETDEV_CHANGE
 };

 enum rdma_port_space {
@@ -328,4 +329,10 @@ void rdma_leave_multicast(struct rdma_cm
  */
 void rdma_set_service_type(struct rdma_cm_id *id, int tos);

+enum  rdma_ha_mode {
+	RDMA_ALIGN_WITH_NETDEVICE = 1
+};
+
+int rdma_set_high_availability_mode(struct rdma_cm_id *id, enum rdma_ha_mode mode);
+
 #endif /* RDMA_CM_H */
Index: linux-2.6.26-rc2/drivers/infiniband/core/cma.c
===================================================================
--- linux-2.6.26-rc2.orig/drivers/infiniband/core/cma.c	2008-05-15 14:48:44.000000000 +0300
+++ linux-2.6.26-rc2/drivers/infiniband/core/cma.c	2008-05-15 16:30:42.000000000 +0300
@@ -143,6 +143,7 @@ struct rdma_id_private {
 	u32			qp_num;
 	u8			srq;
 	u8			tos;
+	enum rdma_ha_mode	ha_mode;
 };

 struct cma_multicast {
@@ -1523,6 +1524,19 @@ void rdma_set_service_type(struct rdma_c
 }
 EXPORT_SYMBOL(rdma_set_service_type);

+int rdma_set_high_availability_mode(struct rdma_cm_id *id, enum rdma_ha_mode mode)
+{
+	struct rdma_id_private *id_priv;
+
+	if (mode != RDMA_ALIGN_WITH_NETDEVICE)
+		return -EINVAL;
+
+	id_priv = container_of(id, struct rdma_id_private, id);
+	id_priv->ha_mode = mode;
+	return 0;
+}
+EXPORT_SYMBOL(rdma_set_high_availability_mode);
+
 static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec,
 			      void *context)
 {