C:send
lastmod 2017-01-11created 2017-01-111 edits
Send Buffer Information
Under normal conditions, send() returns once it has copied all the data into the kernel's send buffer, regardless of whether the data has been transmitted (http://stackoverflow.com/questions/5407182/blocking-sockets-when-exactly-does-send-return). The send buffer is a buffer in the kernel where data that has not yet been ACKed accumulates. Under normal conditions this send buffer is 8K on Windows and 16K on Linux. This value can be adjusted with SO_SNDBUF (http://brucekim.egloos.com/1459382).
With a blocking mode socket, send() can also block; the man page describes it as follows.
When the message does not fit into the send buffer of the socket, send normally blocks, unless the socket has been placed in non-blocking I/O mode.
This means that if you ask a blocking socket to send data that exceeds the send buffer, it blocks (http://kukuta.tistory.com/119). To test this situation, keep calling send() with large data using the Python code above. Before long you can see send() block.
I thought I could use the fact that send() blocks to check on the sender side whether data was sent successfully. However, as described above, the point at which send() returns and the point at which send() returns an error are different, so from the application's point of view there is no way to know up to which data was sent successfully. The situations in which errors occur are well described at http://www.stratus.com/blog/openvos/?p=749. The conclusion of that link is that TCP cannot guarantee data delivery, so you must use an application level ACK.
At this point I should sort out send()'s return value, errno, and socket options.
- SO_SNDTIMEO sets the timeout for send(); when a timeout occurs, send() returns -1 and sets errno to EAGAIN (=EWOULDBLOCK).
- If send() timed out because the TCP send buffer was full, errno becomes ETIMEOUT on the next send() call.
- If a reset comes back after send(), errno becomes ECONNRESET on the next send() call.
- If an interrupt occurs during send(), errno becomes EINTR.
When sending structures delimited by a length field over TCP, I considered the following.
- Can the sender notice a transmission failure caused by a network disconnection?
If the receiver cannot receive data until the sender fills the send buffer, the sender's send() will block and then time out. This timeout can be treated as a problem on the receiver side.
- Can the receiver notice a transmission failure caused by a network disconnection?
The sender sends a message periodically; if this message is not received, it is treated as a network disconnection.
- Can the sender distinguish structures sent successfully from those that failed?
It cannot, because the sender assumes it has sent everything.
- Can the receiver distinguish success and failure per structure?
The biggest problem is when one structure fails halfway through transmission and the remaining half is filled by another structure on receipt. Then the receiver loses any way to delimit structures (it may read a garbage field as the length field), and you're doomed. However, if the receiver drops the connection entirely when it times out before fully receiving a structure (this timeout is separate from the TCP timeout), it seems this could work. An application level ACK is also possible, but then another thread must be created, and a separate queue structure is needed for the responsiveness of the send function.