Monthly Archives: October 2010

TCP and Fragmented Messages

Protocols

A protocol is a set of rules that define how digital messages have to be structured and how to exchange these messages between computers.

When two applications are communicating they usually use a protocol stack, the most popular one being TCP/IP. A widespreadly used application protocol on top of TCP for example is SMTP, that transfers your email messages to the inbox of the receiver.

TCP/IP

The big picture:

  • Application Layer (e.g. your application protocol to implement)
  • Transport Layer (TCP)
  • Internet Layer (IP)
  • Network Layer (e.g. Ethernet, WiFi)

So when implementing an application protocol to be able to communicate with your counterpart you have to understand TCP.

In plain C you use send() to send out a message and on the other side recv() to receive data:


  int send(int socket, char *message, int length, int flags);
  int recv(int socket, char *message, int length, int flags);

TCP is a stream protocol

And now comes the first and most important thing to understand: TCP is a stream protocol.

Application programmer tend to think in messages: but being a stream protocol TCP has no implicit message boundaries. The sender writes bytes to the stream with send() and the receiver reads bytes from the stream with recv().

Fragments, Blocking, and Timeouts

When receiving data with recv() be prepared that your receive the message in fragments. If the sender calls send() subsequently you could also receive data of more then one message at once. It’s a stream – there is no one-to-one relationship between send() and receive().

recv() will block until at least one byte and at most length (see function prototype above) bytes are received.

Since there are no message boundaries you need to know the length of the message to receive. What you do to reassemble the fragments into a message is:

        int numLeft = lengthOfMessage;
        int len = 0;
        int numBytes;
        do {
            numBytes = recv(socket, buf + len, numLeft, 0);
            len += numBytes;
            numLeft -= numBytes;
        } while(numBytes > 0 && len < lengthOfMessage);

        //check result of recv for closed socket (numBytes == 0) and errors (numBytes < 0)
        //if no errors, then...

To avoid that recv() blocks in case of a failure on the sender side, one configures a timeout or makes a check with select() wether the socket is ready for I/O.

Splitting a Message into Header and User Data

Since most messages have a variable length usually a message on the application protocol layer is divided into a header and user data. The header has a fixed size and includes information about the length of the message.

So you first receive the header (fixed size), extract the message size information and then receive in a second loop the real user data.

Alternatively some protocols are using delimiters to mark message boundaries.

TCP/IP in C# or Java

TCP/IP in C# or Java works the same as in C, it’s even more obvious, because you are asking the socket explicitly for streams (GetStream() in C# respectively getInputStream() in Java)!

Summary

When developing network routines there are several things to get right, communication threads, error handling, closing operating system resources, and so on. But avoid the most basic TCP beginner mistake and don’t assume that there are message boundaries. And don’t aggressivly blame the broken network or other developers to send data in fragments or as once happened to me: it doesn’t make a good impression 🙂