head 1.48; access; symbols LMTP2NNTP_1_4_1:1.48 LMTP2NNTP_1_4_0:1.48 SIO_0_9_3:1.48 SA_1_2_6:1.48 SA_1_2_5:1.48 SA_1_2_4:1.48 SA_1_2_3:1.48 LMTP2NNTP_1_3_0:1.48 LMTP2NNTP_1_3b2:1.48 SA_1_2_2:1.48 LMTP2NNTP_1_3b1:1.48 SA_1_2_1:1.48 LMTP2NNTP_1_3a3:1.48 LMTP2NNTP_1_3a2:1.48 LMTP2NNTP_1_3a1:1.48 SA_1_2_0:1.48 SA_1_1_0:1.48 SIO_0_9_2:1.48 SIO_0_9_1:1.48 LMTP2NNTP_1_2_0:1.48 LMTP2NNTP_1_2b4:1.48 LMTP2NNTP_1_2b3:1.48 LMTP2NNTP_1_2b2:1.48 LMTP2NNTP_1_2b1:1.47 LMTP2NNTP_1_2a8:1.47 SIO_0_9_0:1.47 LMTP2NNTP_1_2a7:1.47 SA_1_0_5:1.47 SA_1_0_4:1.47 SA_1_0_3:1.47 SA_1_0_2:1.47 SA_1_0_1:1.46 SA_1_0_0:1.46 SA_0_9_3:1.45 SA_0_9_2:1.36 LMTP2NNTP_1_2a6:1.34 LMTP2NNTP_1_2a5:1.34 SA_0_9_1:1.33 LMTP2NNTP_1_2a4:1.33 LMTP2NNTP_1_2a3:1.33 SA_0_9_0:1.33 LMTP2NNTP_1_2a1:1.31 LMTP2NNTP_1_1_1:1.31 LMTP2NNTP_1_1_0:1.31 LMTP2NNTP_1_1b4:1.31 LMTP2NNTP_1_1b3:1.31 LMTP2NNTP_1_1b2:1.24 LMTP2NNTP_1_1b1:1.24 SA_0_0_0:1.1.1.1 VENDOR:1.1.1; locks; strict; comment @# @; 1.48 date 2003.02.07.20.47.51; author rse; state Exp; branches; next 1.47; 1.47 date 2002.11.07.13.05.43; author rse; state Exp; branches; next 1.46; 1.46 date 2002.10.31.07.17.46; author rse; state Exp; branches; next 1.45; 1.45 date 2002.10.30.18.46.22; author rse; state Exp; branches; next 1.44; 1.44 date 2002.10.30.08.43.16; author rse; state Exp; branches; next 1.43; 1.43 date 2002.10.27.07.56.31; author rse; state Exp; branches; next 1.42; 1.42 date 2002.10.27.07.55.18; author rse; state Exp; branches; next 1.41; 1.41 date 2002.10.26.18.28.15; author rse; state Exp; branches; next 1.40; 1.40 date 2002.10.26.18.01.24; author rse; state Exp; branches; next 1.39; 1.39 date 2002.10.26.17.59.30; author rse; state Exp; branches; next 1.38; 1.38 date 2002.10.26.15.45.32; author rse; state Exp; branches; next 1.37; 1.37 date 2002.10.25.19.53.22; author rse; state Exp; branches; next 1.36; 1.36 date 2002.10.11.15.28.18; author rse; state Exp; branches; next 1.35; 1.35 date 2002.09.22.07.36.43; author rse; state Exp; branches; next 1.34; 1.34 date 2002.04.01.12.22.22; author rse; state Exp; branches; next 1.33; 1.33 date 2002.01.02.13.54.18; author rse; state Exp; branches; next 1.32; 1.32 date 2002.01.02.13.20.11; author rse; state Exp; branches; next 1.31; 1.31 date 2001.10.31.20.07.22; author rse; state Exp; branches; next 1.30; 1.30 date 2001.10.31.19.52.52; author rse; state Exp; branches; next 1.29; 1.29 date 2001.10.31.13.11.25; author rse; state Exp; branches; next 1.28; 1.28 date 2001.10.31.09.46.06; author rse; state Exp; branches; next 1.27; 1.27 date 2001.10.26.10.57.28; author rse; state Exp; branches; next 1.26; 1.26 date 2001.10.24.12.24.55; author rse; state Exp; branches; next 1.25; 1.25 date 2001.10.23.12.02.31; author rse; state Exp; branches; next 1.24; 1.24 date 2001.10.10.15.54.05; author rse; state Exp; branches; next 1.23; 1.23 date 2001.10.10.15.51.38; author rse; state Exp; branches; next 1.22; 1.22 date 2001.10.10.09.21.27; author rse; state Exp; branches; next 1.21; 1.21 date 2001.10.10.09.05.18; author rse; state Exp; branches; next 1.20; 1.20 date 2001.10.10.07.57.29; author rse; state Exp; branches; next 1.19; 1.19 date 2001.10.10.07.42.26; author rse; state Exp; branches; next 1.18; 1.18 date 2001.10.09.18.55.47; author rse; state Exp; branches; next 1.17; 1.17 date 2001.10.09.18.34.12; author rse; state Exp; branches; next 1.16; 1.16 date 2001.10.09.18.27.13; author rse; state Exp; branches; next 1.15; 1.15 date 2001.10.09.18.26.12; author rse; state Exp; branches; next 1.14; 1.14 date 2001.10.08.15.17.47; author rse; state Exp; branches; next 1.13; 1.13 date 2001.10.07.15.21.17; author rse; state Exp; branches; next 1.12; 1.12 date 2001.10.07.14.17.13; author rse; state Exp; branches; next 1.11; 1.11 date 2001.10.06.11.05.51; author rse; state Exp; branches; next 1.10; 1.10 date 2001.10.06.11.02.09; author rse; state Exp; branches; next 1.9; 1.9 date 2001.10.05.21.37.30; author rse; state Exp; branches; next 1.8; 1.8 date 2001.10.05.21.37.05; author rse; state Exp; branches; next 1.7; 1.7 date 2001.10.05.20.52.15; author rse; state Exp; branches; next 1.6; 1.6 date 2001.10.05.13.18.18; author rse; state Exp; branches; next 1.5; 1.5 date 2001.10.05.13.17.36; author rse; state Exp; branches; next 1.4; 1.4 date 2001.10.05.11.55.51; author rse; state Exp; branches; next 1.3; 1.3 date 2001.10.05.11.40.22; author rse; state Exp; branches; next 1.2; 1.2 date 2001.10.04.11.25.44; author rse; state Exp; branches; next 1.1; 1.1 date 2001.10.02.13.27.45; author rse; state Exp; branches 1.1.1.1; next ; 1.1.1.1 date 2001.10.02.13.27.45; author rse; state Exp; branches; next ; desc @@ 1.48 log @Add internal address resolving support via the new IEEE Std 1003.1g-2000 ("POSIX.1") getaddrinfo(3) API. This especially provides more portable IPv6 address resolving. @ text @ _ ___ ____ ____ ____ |_|_ _ / _ \/ ___/ ___|| _ \ ___ __ _ _|_||_| | | | \___ \___ \| |_) | / __|/ _` | |_||_|_| | |_| |___) |__) | __/ \__ \ (_| | |_|_|_| \___/|____/____/|_| |___/\__,_| OSSP sa - Socket Abstraction TODO The following items still have to be done: -NONE- CANDO The following items can be done if wished and time permits: o Add support for more socket options, for instance SO_BROADCAST for enabling permission to transmit broadcast messages. o what about actually written bytes in case of sa_writef? Should we internally retry somehow? o provide Perl glue module SA. o provide "satool", a command line interface to the OSSP sa API. Can optionally use readline or libedit, etc. o support for SSL/TLS via small abstraction layer on top of OpenSSL and gnutls libraries. o Consistently use PF_XXX instead of AF_XXX whenever not directly address related things are done in order to even closer conform to POSIX. o DNS Resolving and multiple results. On DNS resolving name, multiple result addresses can be returned. Either because of Round-Robin entries, multi-homed hosts or because of hosts talking both IPv4 and IPv6, etc. Each address has to be tried on socket+connect. What to do? For some hints see: http://www.v6.wide.ad.jp/Presentations/ai3-penang0010-v6programming/mgp00015.html o Nagle's Algorithm and Flushing of Output Buffers. The kernel performs Nagle's Algorithm (see RFC 896 and search for "nagle algorithm" on www.whatis.com [currently http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci754347,00.html] on his internal output buffers. Although we flush our user-space output buffers only if we really perform a read(2) (and not if the read is going to be served entirely from the local buffer) this does not mean that really every character the remote has already sent is also already read by us. Because the kernel also has a read buffer. Optimal behaviour would be that we flush out output buffer only if the read(2) would block. OTOH flushing out buffers means performing a write(2) and this again is buffered in the kernel, too. So performing an optimal read->write->read->write->... sequence is very complex and non-trivial to implement. Especially because even using Nagle's Algorithm always is not the right choice (see http://www.w3.org/Protocols/HTTP/Performance/Nagle/ for details), especially when it comes to pipelining protocols. o Support for writev(2). This can be done by internally switching to always use writev(2), providing an emulation for writev(2) ala Pth and by basing the write calls always on writev. o Kernel Read/Write Buffer Adjustments. BSD Sockets usually provide (see setsockopt(2)): SO_SNDBUF set buffer size for output SO_RCVBUF set buffer size for input SO_SNDLOWAT set minimum count for output SO_RCVLOWAT set minimum count for input This would mean that we could also allow the control of the kernel buffers via SA_BUFFER_KREAD SA_BUFFER_KWRITE Unfortunately the whole kernel buffer issue is very complex, because according to STEVENS there are both minimum and maximum sizes and both borders heavily depend on the currently used protocol (TCP or UDP) and the MTU of the underlying network, etc. This all together seems like opening a can of worms if we provide SA_BUFFER_K{READ,WRITE}. @ 1.47 log @remember issue @ text @a35 7 o Support for newer DNS Resolving Functions. In IPv6 land there are usually the newer DNS resolving functions getaddrinfo(3) and getnameinfo(3). Perhaps we should support these in favor of gethostbyname(3) and gethostbyaddr(3) because the chance is higher that they support IPv6 in case we are in IPv6 land. @ 1.46 log @polish for OSSP sa 1.0.0 release @ text @d19 3 @ 1.45 log @puhhh... finally finish test suite @ text @d10 4 a13 1 ---- d16 2 a17 1 ----- @ 1.44 log @now done @ text @a11 2 o more test cases to cover full API and especially the client/server usage @ 1.43 log @remember before working on it @ text @a13 3 o SA_OPTION_REUSE{ADDR,PORT} has to be done _before_ sa_bind(3) but currently the underlying socket is still not allocated there. @ 1.42 log @move to cando section @ text @d14 3 @ 1.41 log @finish documentation @ text @a11 2 o what about actually written bytes in case of sa_writef? d16 3 @ 1.40 log @remember already started sub-project @ text @a15 2 o more documentation to fully describe API @ 1.39 log @cleanup source tree @ text @d21 2 @ 1.38 log @strip trailing whitespaces @ text @d1 5 d7 1 a7 2 TODO ---- d9 2 a10 2 - more test cases to cover full API and especially the client/server usage - more documentation to fully describe API d12 1 a12 2 CANDO ----- d14 75 a88 68 o provide "satool", a command line interface to the OSSP sa API. Can optionally use readline or libedit, etc. o support for SSL/TLS via small abstraction layer on top of OpenSSL and gnutls libraries. o Consistently use PF_XXX instead of AF_XXX whenever not directly address related things are done in order to even closer conform to POSIX. o Support for newer DNS Resolving Functions. In IPv6 land there are usually the newer DNS resolving functions getaddrinfo(3) and getnameinfo(3). Perhaps we should support these in favor of gethostbyname(3) and gethostbyaddr(3) because the chance is higher that they support IPv6 in case we are in IPv6 land. o DNS Resolving and multiple results. On DNS resolving name, multiple result addresses can be returned. Either because of Round-Robin entries, multi-homed hosts or because of hosts talking both IPv4 and IPv6, etc. Each address has to be tried on socket+connect. What to do? For some hints see: http://www.v6.wide.ad.jp/Presentations/ai3-penang0010-v6programming/mgp00015.html o Nagle's Algorithm and Flushing of Output Buffers. The kernel performs Nagle's Algorithm (see RFC 896 and search for "nagle algorithm" on www.whatis.com [currently http://searchnetworking.techtarget.com/sDefinition/0,,sid7_gci754347,00.html] on his internal output buffers. Although we flush our user-space output buffers only if we really perform a read(2) (and not if the read is going to be served entirely from the local buffer) this does not mean that really every character the remote has already sent is also already read by us. Because the kernel also has a read buffer. Optimal behaviour would be that we flush out output buffer only if the read(2) would block. OTOH flushing out buffers means performing a write(2) and this again is buffered in the kernel, too. So performing an optimal read->write->read->write->... sequence is very complex and non-trivial to implement. Especially because even using Nagle's Algorithm always is not the right choice (see http://www.w3.org/Protocols/HTTP/Performance/Nagle/ for details), especially when it comes to pipelining protocols. o Support for writev(2). This can be done by internally switching to always use writev(2), providing an emulation for writev(2) ala Pth and by basing the write calls always on writev. o Kernel Read/Write Buffer Adjustments. BSD Sockets usually provide (see setsockopt(2)): SO_SNDBUF set buffer size for output SO_RCVBUF set buffer size for input SO_SNDLOWAT set minimum count for output SO_RCVLOWAT set minimum count for input This would mean that we could also allow the control of the kernel buffers via SA_BUFFER_KREAD SA_BUFFER_KWRITE Unfortunately the whole kernel buffer issue is very complex, because according to STEVENS there are both minimum and maximum sizes and both borders heavily depend on the currently used protocol (TCP or UDP) and the MTU of the underlying network, etc. This all together seems like opening a can of worms if we provide SA_BUFFER_K{READ,WRITE}. @ 1.37 log @remember issue @ text @d36 1 a36 1 d58 1 a58 1 @ 1.36 log @now fixed, too. @ text @d11 3 @ 1.35 log @remember @ text @a4 8 Solaris 2.6: ./libtool --mode=compile --quiet /usr/opkg/bin/cc -c -O -pipe -I. l2_env.c In file included from l2_p.h:43, from l2_env.c:30: l2_ut_sa.h:161: parse error before "socklen_t" l2_ut_sa.h:163: parse error before "socklen_t" make[2]: *** [l2_env.lo] Error 1 a6 1 @ 1.34 log @remember issue @ text @d5 8 @ 1.33 log @todo @ text @d8 1 d11 3 @ 1.32 log @first cut for PF_XXX usage @ text @d5 1 a5 1 - more test cases to cover full API @ 1.31 log @move two things from TODO to CANDO and describe in detail @ text @d11 3 @ 1.30 log @now provided by sa_option() @ text @a4 5 - on resolving multiple addresses can be returned (especially one IPv4 and one IPv6!) and each one has to be tried on socket+connect for instance. What to do? See also http://www.v6.wide.ad.jp/Presentations/ai3-penang0010-v6programming/mgp00015.html - support for new getaddrinfo and getnameinfo (IPv6!) d10 15 @ 1.29 log @remember the issue now that we did our best @ text @a9 1 - API function for blocking/non-blocking I/O mode @ 1.28 log @remember issue here, too. @ text @a4 12 Oh, finally, a performance issue (that probably doesn't matter for lmtp2nntp): sa_rc_t sa_read(sa_t *sa, char *cpBuf, size_t nBufReq, size_t *nBufRes) always flushes the write portion of the outgoing stream, even if the read is going to be entirely from the local buffer. This is generally undesirably behavior, since you can trigger nagle's algorithm if you're a command/response server with pipelining like LMTP. Generally, you want to flush the write portion whenever you're about to block on a read, but not necessarily before. (Of course, the client-side of a pipelining client is considerably more complex.) d16 18 @ 1.27 log @errr.. I meant blocking, not buffering @ text @d5 12 @ 1.26 log @remember multiple return issue on resolving @ text @d10 1 a10 1 - API function for unbuffered/buffered I/O mode @ 1.25 log @remember issue @ text @d5 5 @ 1.24 log @update TODO @ text @d5 1 @ 1.23 log @update TODO @ text @d5 3 @ 1.22 log @Revert my idea of using an L2-style error handling also in SA, because this is not possible in a consistent way due to the multiple objects and the fact that we cannot remember anything without an object (in L2 an environment object will be introduced soon, so there is it is a different game). So we use a plain sa_error() which is now similar to strerror(3). @ text @a4 8 o Provide a full test suite for the whole API. - inet://0.0.0.0:0 -> inet://0.0.0.0:514 - inet://127.0.0.1:514 -> inet://127.0.0.1:514 - inet://localhost:syslog#udp -> inet://127.0.0.1:514 - inet://localhost:smtp -> inet://127.0.0.1:25 - unix:/tmp/socket -> unix:/tmp/socket @ 1.21 log @Woohoooo, fully revamp formatting stuff: - evolve sa_mvsnprintf() into an sa_mvxprintf() which 1. is output independent 2. support especially no-output mode (length determination only) 3. performs better by output plain text in largest possible chunks 4. does output operation only once in the loop (smaller code-size) - write a new sa_mvsnprintf() for buffer output based on sa_mvxprintf() - change sa_writef() from brain-dead and problematic "first format anything into temporary buffer following by a single write" to the expected "format on-the-fly into the SA_BUFFER_WRITE and if not enough space exists, write directly to the socket". @ text @a12 2 o Use sa_errorinfo() internally everywhere where it is reasonable. @ 1.20 log @cleanup @ text @a4 5 o Perhaps move 1024-buffer of writef into sa_t and make adjustable or even better: provide a sa_mvsnprintf() variant which directly does the sa_write calls and this way the whole buffering is the SA_BUFFER_WRITE. This would be exactly what the user would expect. @ 1.19 log @Change sa_rc_t sa_buffers (sa_t *sa, size_t rsize, size_t wsize); to sa_rc_t sa_buffer (sa_t *sa, sa_buffer_t id, size_t size); in order to make this API function more flexible. @ text @d2 10 a11 5 - support for writev(2) - perhaps move 1024-buffer of writef into sa_t and make adjustable - SA_BUFFER_KREAD SA_BUFFER_KWRITE - full test suite for whole API d17 32 a48 1 - use sa_errorinfo() internally @ 1.18 log @remember error issue @ text @d4 1 a4 4 - sa_buffer(sa, , size) SA_BUFFER_UREAD SA_BUFFER_UWRITE SA_BUFFER_KREAD a5 1 SA_BUFFER_WRITEF @ 1.17 log @Change return code semantic for timeouts from "rv == SA_ERR_SYS && errno == ETIMEDOUT" to a dedicated timeout return value "rv == SA_ERR_TMT". @ text @d16 1 @ 1.16 log @update TODO @ text @a1 1 - perhaps change rv=SA_ERR_SYS+errno=ETIMEDOUT to rv=SA_ERR_TIMEOUT; @ 1.15 log @support relative paths for 'unix:path' @ text @d3 1 a4 3 - support registry sub-library for system call override - full test suite for whole API - support for writev(2) d11 1 a11 2 - test suite: @ 1.14 log @remember TODOs @ text @a12 1 - support for relative pathnames in unix:path @ 1.13 log @Implement (still fully untested!) system call overriding support through a new function sa_syscall(). Anybody here who finds predestinated to test this out in depth? @ text @d13 9 @ 1.12 log @update TODO @ text @a12 15 - sa_syscall(sa, |, func, ctx); SA_SYSCALL_CTX SA_SYSCALL_SOCKET SA_SYSCALL_BIND SA_SYSCALL_CONNECT SA_SYSCALL_LISTEN SA_SYSCALL_ACCEPT SA_SYSCALL_READ SA_SYSCALL_WRITE SA_SYSCALL_RECVFROM SA_SYSCALL_SENDTO SA_SYSCALL_SHUTDOWN SA_SYSCALL_CLOSE : @ 1.11 log @remember writev issue @ text @a5 2 - attention: timeout on accept() can be not what is wishes, i.e., we need specific timeouts d7 20 @ 1.10 log @remember timeout issue @ text @d8 1 @ 1.9 log @remember test suite issue @ text @d6 2 @ 1.8 log @update TODO @ text @d5 1 @ 1.7 log @Woohhhooo: Second major revamp of Socket Abstraction (SA) library. This time is gets really close to a 100% clean and polished library. @ text @d2 3 a4 2 ETIMEDOUT -> SA_ERR_TIM; 1025 buffer von writef in sa_t aufnehmen und variablen machen! @ 1.6 log @no timeout is already implemented @ text @d1 4 @ 1.5 log @Support fully unbuffered I/O and make this the default. @ text @a0 4 TODO: timeout == 0 -> fully blocking @ 1.4 log @remember one more thing @ text @a2 1 buffer == 0 -> fully unbuffered @ 1.3 log @Major revamp of SA library in order to support Unix Domain sockets. - "{tcp,udp}://host:port" addresses are now "inet://host:port" addresses - "unix:/path/to/socket" is the address for Unix Domain sockets - sa_type() was introduced which sets socket type to stream or datagram @ text @a1 7 Goals: 1. struct sockaddr * abstraction 2. ssize_t abstraction 3. cache socket descriptors weil kein close existiert 4. jede funktion hat timeout-possibility (connect_smart!) 5. optional readline and buffering functionality @ 1.2 log @Change sa_addr_t API: - there is now a constructor and destructor - renamed sa_x2x functions to sa_addr_x2x @ text @a12 4 - Unix domain sockets support - aufpassen dasz fuer read/write STREAM socket ist - aufpassen dasz fuer readfrom/writeto DGRAM socket ist @ 1.1 log @Initial revision @ text @a15 1 - destructor fuer sa_addr_t? @ 1.1.1.1 log @Import first cut of OSSP SA @ text @@