September 1998, Issue 98

TCP/IP Networking

---

TCP/IP NETWORKING PRIMER

Recall that TCP/IP implementations use the socket API for application programs to interface to the stack (see "Network Communication," INK 96). In a nutshell, when two programs want to establish a TCP/IP connection, they set up a socket which is identified with an Internet address and a port.

Once the connection is set up, the programs write data to the socket as if it were a file, and the data comes out the other end of the socket, where the other program reads it like a file. The protocol stack treats the TCP/IP as an unstructured bytestream, so it’s easy to program for TCP/IP.

But as you probably know, the actual network interface between computers usually consists of a LAN or serial interface, with data being transferred in packets. Let’s look at what goes on in the TCP/IP stack to give us this illusion of a connection.

The protocol stack for TCP/IP is illustrated in Figure 1. At the top, we have the application layer, which is followed by the transport and Internetwork layers. The network interface and the network are at the bottom.

Figure 1—Each layer in a TCP/IP stack has a specific function and adds a header to the data from the application layer.

The application program, in the application layer, is considered part of the protocol stack. The application usually implements some kind of protocol on top of the unstructured bytestream provided by TCP/IP.

For example, Internet mail uses the Simple Mail Transport Protocol (SMTP) to exchange E-mail messages between different hosts on the Internet. You may also be familiar with the Hypertext Transport Protocol (HTTP) that Web browsers and servers use to communicate.

Note that application-layer protocols are implemented in the application and that the interface between the application layer and the transport layer is the socket API. The rest of the protocol stack—the transport layer through the network interface—is usually implemented in the OS because its implementation is the same for any application.

The transport layer implements the bytestream abstraction of the transmission control protocol (TCP) over the packet-switched Internetwork protocol (IP).

TCP implements what networking folks call a reliable virtual circuit. It is reliable because the implementation makes sure that the data, once received, is not corrupted by noise, arrives in order, and nothing is missing or duplicated.

A virtual circuit is kind of like a phone connection. Once the connection is made, it is maintained until disconnected.

TCP implements this reliable virtual circuit by maintaining a state on each end of the connection. There is state information for each active (or connected) port. The status information maintains the connection state (i.e., whether the circuit is being set up or torn down and whether it is connected).

Sequence pointers are kept to indicate which byte of the stream has been received, acknowledged, and delivered to the receiver. The transmitter also tracks which byte it sent, how much data is left to send, and the last byte acknowledged by the receiver.

Each time one side of the connection has data to send (TCP is full-duplex, by the way), it bundles up the data into a packet and adds a TCP header to the packet.

The packet header includes an end-to-end checksum to make sure that the TCP packet is received intact, a copy of the sequence pointer of the sent data, and a copy of the sequence number for the data it last received, which serves as the acknowledgment. Including the acknowledgement for the data received, while transmitting data in the other direction, is referred to as piggybacking the acknowledgment.

The header also contains the destination port number this TCP packet goes to and the source port number from which it was sent. Figure 2 shows how the various sequence pointers are related.

Once the TCP header has been added, the TCP packet is passed to the Internetwork layer. This layer gets packets to and from other hosts using the network interface.

This layer adds an IP header to the TCP packet. This header contains the destination host address and the source host address. These addresses along with the port numbers from the TCP header let us uniquely identify which TCP connection this packet belongs to as well as the direction it needs to go.

The Internet layer needs to send the packet out over a network using one of the one or more network interfaces. Internet packets can be up to 8 KB long, but most network interfaces can’t handle packets of that size. For example, an Ethernet interface can only handle packets 1506 bytes long.

Therefore, the Internet layer often needs to fragment the IP packet into smaller packets. It splits the packet into smaller chunks and copies the IP header into each one.

A field in the IP header tells at what offset in the original packet the fragment packet belongs. The receiving host’s Internet layer reassembles the fragments into the original packet before passing it on to the receiving transport layer.

For the Internet to work, we need to be able to connect various networks together via special hosts that have more than one network interface. These hosts are called routers or Internet gateways.

On a router, the Internet-layer implementation also needs to decide which network interface a packet needs to be sent on. A routing table does this job.

Now we’re at the business end of the protocol stack, the network interface layer, which has the device drivers for the network devices. A network device may be an Ethernet card, a serial port, or wireless Ethernet interface that connects the host to the network.

Probably the most common network device is the Ethernet interface. Its versatile LAN architecture can be connected to twisted pair, coax, or fiber-based media. An Ethernet card usually has some memory in which the outgoing Ethernet packet is constructed and received. The actual transmitting and receiving is done with hardware on the card.

The network interface driver copies the packet it received from the Internet layer into the Ethernet card’s memory, adds an Ethernet header to it, and tells the card to send the packet. When the card transmits the Ethernet packet, it interrupts the network interface driver. Of course, the card also interrupts if a packet was received from the Ethernet.

The network interface usually communicates with the network driver queues. The transmit and receive queues decouple the upper level network code from the interrupt service code for the driver. In a real-time application, this task is especially important because we don’t want to spend a lot of time in the interrupt service routine.

The receive interrupt routine simply pulls the packet from the Ethernet card’s memory, puts it in the receive queue, and signals the upper level code. The transmit interrupt routine checks the transmit queue. If it has something to send, it pulls off the packet from the queue, copies it to the card’s memory, and starts the transmission.

Newer cards—especially PCI-based cards that can do bus mastering and access system memory—use DMA to send the packets directly from system memory. For these cards, the network driver simply maintains a list of buffer pointers that point to receive and transmit buffer memory. These cards don’t require the CPU to copy the data around during interrupt time. Figure 3 shows the basic idea.

Figure 3—Bus-mastering Ethernet controllers can DMA the packets directly from system memory. The device driver simply sets up a DMA chain to tell the controller where to find the packets.