[ofa-general] ***SPAM*** Re: [PATCH] libsdp: enable fallback to TCP for nonblocking sockets

[Yossi Etigin]

Hi Amir,

The single case in which we block connect() here (and only on SDP, which
is rather fast) is the case that is currenlty not supported anyway. It can
also be configurable.
 Anyway, we have a client which uses non-blocking sockets and really needs
that feature. How about putting this to OFED now and writing something better
later on?

--Yossi


Amir Vadai wrote:
> See below
> 
> On Thu, 2008-08-21 at 19:49 +0300, Yossi Etigin wrote:
>> Hi Amir,
>>
>> What you suggesting is to replace almost all socket functions, and I
>> don't think that this is good either.
> I agree - but to break the non-blocking semantics is worse.
> 
>> It would be write(), send(), recv(), sendto(), recvfrom(), sendmsg(),
>> recvmsg(), and also need to change select() (to not return when
>> fallback
>> happens if SDP fails), and maybe also poll(). libsdp tries to avoid
>> the fast path.
> I don't see another option. We could have a #ifdef to enable the user
> to choose - non blocking support or cleaner fast-path.
>> Besides, how do we know when to do fallback - can we safely assume
>> that if some socket operation fails, then it happened because
>> connect() failed?
>>From a brief look at connect man page, they say we should use select for
> writing on the socket. after select indicates writability, use
> getsockopt to determine whether connect() completed successfully or not.
>> Anyway, if I understand correctly, you suggest something like:
>>
>> int connect(fd, ...)
>> {
>>         ...
>>         set_state(fd, SDP)
>>         ...
>> }
>>
>>
>> int read(int fd, ...)
>> {
>>         int res = socket_funcs.read(shadow_fd(fd), ...);
>>         if (res < 0 && errno != EAGAIN && sock_state(fd) == SDP) {
>>                 sock_state = TCP;
>>                 sockt_funs.connect(fd,...);
>>                 close(shadow_fd(fd));
>>                 errno = EAGAIN;
>>         }
>>         return res;
>> }
>>
>>
> ... again, I don't like it too - but I don't think we should block
> connect when the user asks not to.
> - Amir.
>> --Yossi
>>
>> Amir Vadai wrote:
>>> Yossi Hi,
>>>
>>> I think that breaking the semantic of non blocking socket is a bad
>> idea.
>>> There is a solution that won't break this semantics:
>>>
>>> 1. User app calls connect().
>>>       - libsdp try to connect through sdp.
>>> 2. User app try another operation on the socket (e.g read/write)
>>>       - if sdp connection established successfully - great
>>>       - if sdp still not established - return -EAGAIN. This is the
>>> same behaviour as if the tcp connection wasn't connected yet.
>>>       - if sdp timedout - return -EAGAIN and initiate TCP connect.
>>>       - if tcp connection established - use it
>>>       - if tcp connection timedout - return error.
>>>
>>> Maybe we could optimize it and initiate a tcp connection in parallel
>>> with the sdp connection and use it only when the sdp connect is
>>> timedout.
>>>
>>> I will add only the second patch (the debug print fix).
>>>
>>> - Amir
>>>
>>>
>>
>>
>