[openib-general] Connection Manager Abstraction proposition -header file
Tom Tucker
tom at ammasso.com
Wed Aug 24 08:07:20 PDT 2005
Guy:
I think we're on the right track. Regarding the 'get-device' function,
the ib_at_route_by_ip service already returns a data structure that
includes the device pointer. This data structure also includes the local
and next hop remote IP addresses which is a good thing. First, it avoids
requiring the connect method to look this data up again. For IB, these
IP addresses are first converted to GID/LID with AT and then to a path
record. For iWARP, these addresses are fine as is, and avoid having the
connect method do a second lookup to find the next hop given the remote
ip address. In other words, pass the ib_at_ib_route structure to the
connect method along with the remote IP address to allow this info to
be reused.
With regard to ib_cma_id, what is the iwarp_id? Is this the remote port?
With regard to listen, it should not require a device pointer because
the
app may in fact be listening on multiple devices. Also, the service_id
does not provide enough information to determine which devices to listen
on. It should be local ip (could be 0 for wild-card), and local port
(service id).
The listen callback function needs to know the device on which the
connection request was received. This could be included in the
ib_cma_id structure.
> -----Original Message-----
> From: openib-general-bounces at openib.org
> [mailto:openib-general-bounces at openib.org] On Behalf Of Guy German
> Sent: Wednesday, August 24, 2005 7:21 AM
> To: openib-general at openib.org
> Subject: [openib-general] Connection Manager Abstraction
> proposition -header file
>
> /*
> * Copyright (c) 2005 Voltaire Inc. All rights reserved.
> *
> * This Software is licensed under one of the following licenses:
> *
> * 1) under the terms of the "Common Public License 1.0" a
> copy of which is
> * available from the Open Source Initiative, see
> * http://www.opensource.org/licenses/cpl.php.
> *
> * 2) under the terms of the "The BSD License" a copy of which is
> * available from the Open Source Initiative, see
> * http://www.opensource.org/licenses/bsd-license.php.
> *
> * 3) under the terms of the "GNU General Public License
> (GPL) Version 2" a
> * copy of which is available from the Open Source Initiative, see
> * http://www.opensource.org/licenses/gpl-license.php.
> *
> * Licensee has the right to choose one of the above licenses.
> *
> * Redistributions of source code must retain the above copyright
> * notice and one of the license notices.
> *
> * Redistributions in binary form must reproduce both the
> above copyright
> * notice, one of the license notices in the documentation
> * and/or other materials provided with the distribution.
> *
> */
>
> /*
> * This header file as a preliminary proposition for a
> connection manager
> * abstraction layer (cma) for IB and iwarp
> * - there is an assumption that iwarp uses the same openib
> qp terminology in
> * the rest of the verbs, and the only place needs
> abstraction is the cm.
> * - This proposition assumes that the address translation
> is done in the cma
> * layer.
> * - The cma also modifies the qp states to init/rtr/rts and
> error as needed.
> * - for calling accept/reject or disconnect on the passive
> side you need to
> * use the cma handle accepted in ib_cma_listen cb.
> * - cma_id is created when calling connect or listen and
> destroyed when
> * accepting disconnected/rejected/unreachable events on
> either active
> * side (connect cb) or passive side (accept cb)
> */
>
> #ifndef IB_CMA_H
> #define IB_CMA_H
>
> #include <linux/socket.h>
>
> enum ib_cma_event {
> IB_CMA_EVENT_ESTABLISHED,
> IB_CMA_EVENT_REJECTED,
> IB_CMA_EVENT_DISCONNECTED,
> IB_CMA_EVENT_UNREACHABLE
> };
>
> enum ib_qos {
> IB_QOS_BEST_EFFORT = 0,
> IB_QOS_HIGH_THROUGHPUT = (1 << 0),
> IB_QOS_LOW_LATENCY = (1 << 1),
> IB_QOS_ECONOMY = (1 << 2),
> IB_QOS_PREMIUM = (1 << 3)
> };
>
> enum ib_connect_flags {
> IB_CONNECT_DEFAULT_FLAG = 0x00,
> IB_CONNECT_MULTIPATH_FLAG = 0x01
> };
>
> /*
> * for ib_cma_get_src_ip - ib_cma_id will have to include
> * the path data received in the request handler
> */
> union ib_cma_id{
> struct ib_cm_id *cm_id;
> u32 iwarp_id;
> };
>
> typedef void (*ib_cma_rarp_handler)(struct sockaddr *src_ip,
> void *context);
> typedef void (*ib_cma_ac_handler)(enum ib_cma_event event,
> void *context);
> typedef void (*ib_cma_event_handler)(enum ib_cma_event event,
> void *context,
> void *private_data);
> typedef void (*ib_cma_listen_handler)(union ib_cma_id *cma_id,
> void *private_data, void
> *context);
>
> struct ib_cma_conn {
> struct ib_qp *qp;
> struct ib_qp_attr *qp_attr;
> struct sockaddr *dst_ip;
> __be64 service_id;
> void *context;
> ib_cma_event_handler cma_event_handler;
> const void *private_data;
> u8 private_data_len;
> u32 timeout;
> enum ib_qos qos;
> enum ib_connect_flags connect_flags;
> };
>
>
> /**
> * ib_cma_get_device - Returns the device to be used according to
> * the destination ip address (this can be detemined according
> * to the local routing table). Call this function before
> * creating the qp. If using link-local IPv6 addresses
> * @remote_address: The destination address for connection
> * @device: The device to use (returned by the function)
> */
> int ib_cma_get_device(struct sockaddr *remote_address,
> struct ib_device **device);
>
>
> /**
> * ib_cma_connect - this is the connect request function, called by
> * the active side. The consumer registers an upcall that will be
> * initiated by the cma with an appropriate connection event
> * notification (established/rejected/disconnected etc)
> * @cma_conn: This structure contains the following
> connection parameters:
> * @qp: qp for establishing the connection
> * @qp_attr: only relevant attributes are used
> * @dst_ip: destination ip address
> * @service_id: destination service id (port)
> * @context: context to be returned in the callback
> * @cma_event_handler: the upcall function for the active side
> * @private_data: private data to be received at the listener upcall
> * @private_data_len: private data length (max 255)
> * @timeout:
> * @qos: Quality os service for the rc
> * @connect_flags: default or multipath connection
> * @cma_id: This returned handle is a union (different in ib
> and iwarp)
> * in ib - it is the cm_id.
> */
> int ib_cma_connect(struct ib_cma_conn *cma_conn,
> union ib_cma_id *cma_id);
>
>
> /**
> * ib_cma_disconnect - this function disconnects the rc. It can be
> * called, by either the passive or active side
> * @qp: the connected qp to disconnect
> * @cma_id: On the active side- this handle is the one returned
> * when ib_cma_connect was called.
> * On the passive side- this handle was accepted in
> cma_listen callback
> */
> int ib_cma_disconnect(struct ib_qp *qp, union ib_cma_id *cma_id);
>
>
> /**
> * ib_cma_sid_listen - this function is called by the passive
> side. It is
> * listening on a the specified port (ib service id) for incomming
> * connection requests
> * @device: ? need to resolve this issue
> * @service_id: service id (port) to listen on
> * @context: user context to be returned in the callback
> * @cm_listen_handler: the listen callback
> * @cma_id: cma handle for the passive side
> */
> int ib_cma_sid_listen(struct ib_device *device, __be64 service_id,
> void *context, ib_cma_listen_handler
> cm_listen_handler,
> union ib_cma_id *cma_id);
>
>
> /**
> * ib_cma_sid_destroy - this functionis is called on the
> passive side, to
> * stop listenning on a certain sevice id
> * @cma_id: the same cma handle received when
> ib_cma_sid_listen was called
> */
> int ib_cma_sid_destroy(union ib_cma_id *cma_id);
>
>
> /**
> * ib_cma_accept - call on the passive side to accept a
> connection request
> * @cma_id: this handle was accepted in cma_listen callback
> * @qp: the connection's qp
> * @private_data: private data to send back to the initiator
> * @private_data_len: private data length
> * @context: user context to be returned in the callback
> * @cm_accept_handler: the cma accept callback - triggered
> when RTU ack
> * received
> */
> int ib_cma_accept(union ib_cma_id *cma_id, struct ib_qp *qp,
> const void *private_data, u8 private_data_len,
> void *context, ib_cma_ac_handler cm_accept_handler);
>
> /**
> * ib_cma_reject - call on the passive side to reject a
> connection request.
> * This call destroys the cma_id, hence when the active side accepts
> * the reject the cma_id is already destroyed.
> * @cma_id: this handle was accepted in cma_listen callback
> * @private_data: private data to send back to the initiator
> * @private_data_len: private data length
> */
> int ib_cma_reject(union ib_cma_id *cma_id, const void *private_data,
> u8 private_data_len);
>
>
> /**
> * ib_cma_get_src_ip - this function performs "rarp", asynchronicly
> * from cma_id to src ip
> * @cma_id: the cma_id will have to include the path data received
> * in the request handler
> * @src_ip: source ip of the initiator
> */
> int ib_cma_get_src_ip(union ib_cma_id *cma_id,
> ib_cma_rarp_handler rarp_handler,
> void *context);
>
> #endif /* IB_CMA_H */
>
> _______________________________________________
> openib-general mailing list
> openib-general at openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit
> http://openib.org/mailman/listinfo/openib-general
>
More information about the general
mailing list