Protocols

A protocol is a set of rules that specify how two devices can communicate with each other. The internet uses many different protocols, and they decide every part of how it works. Without these rules, devices of different kinds would have no way of communicating with each other.

Communication over the internet involves many different technologies. For any particular message, a subset of these technologies is chosen to carry the communication between two applications running on different devices.

Devices within a network communicate and share information about their connections, so that they learn the best paths between networks. Data to be sent is handed to the network, and the devices send it over the best path to the destination. The network devices must work together to carry the data across the whole communications path.

The path chosen has to adapt in real time to changing situations, such as congested or broken links. The devices must also cope with data that is "lost" in the network, or that arrives in a different order from the one in which it was sent. The data may need to be encrypted and decrypted on some parts of the path, or over the whole "end-to-end" path.

The overall system must be able to use a range of different media for each part of the transmission path, and adapt easily as new technologies are developed and added to the internet.

The protocols take the data from the application that wants to send it, and turn it into bits encoded as a suitable optical, electrical or radio signal. This signal is then sent onto the media (any wired or wireless technology that data can be sent over). At the receiving end the reverse happens: the bits are turned back into the data, which is passed to the receiving application.

This complex set of jobs is organised into a series of layers. The data is passed from layer to layer for processing, before it is sent out onto the media.

These layers can be pictured as a set of processes arranged on top of each other to form a stack, the protocol stack. On the sending device, data flows down the stack from the application and out onto the media. At the receiving end, the data flows from the media up the stack, to be delivered to the application on the receiving device.

The TCP/IP protocol suite has four layers. From the top, they are the application layer, the transport layer, the internet layer and the link layer.

• The application layer uses protocols that are designed to work with particular types of application. For example, a web browser uses HTTP, while an email client uses SMTP, together with either IMAP or POP3.
• The transport layer breaks the application data down into segments (TCP) or datagrams (UDP). Sequence numbers are given out, and the source and destination port numbers are added to the header. With TCP, delivery is acknowledged and lost segments are sent again.
• The internet layer (sometimes called the network layer) prepares packets for the internet. The source and destination IP addresses are added at this stage.
• The link layer (sometimes called the data link layer) carries IP packets across each of the separate links that make up the path between client and server. At this level, networking protocols such as Ethernet are used.

When data is sent, each layer of the TCP/IP model accepts data from the layer above (or from a program, for the application layer). It adds the extra information that layer needs to do its job. Sometimes the original data is processed first, for example encrypted, before the extra information is added. The layer then passes the data on to the layer below (or onto the transmission media, for the link layer).

At the receiving end the process happens in reverse. Each layer accepts data from the layer below, inspects and removes the extra information that the matching layer on the sending device added, and passes the data up to the layer above.

So each layer adds its own headers to the data from the layer above. This encapsulation at each layer gives a new structure that wraps up the original data. Each structure has its own name, which depends on the layer that creates it:

The suite is named after its two most important protocols, TCP and IP.

Each type of application is linked to a different protocol, or set of protocols. If you run a web browser and want to communicate with a web server on a distant computer, your browser uses Hypertext Transfer Protocol (HTTP), or Hypertext Transfer Protocol Secure (HTTPS). If you open your email client and send an email, the email client uses Simple Mail Transfer Protocol (SMTP).

If you are running several applications that communicate over the internet at the same time, each one communicates with its own application layer protocol(s) to manage the exchange of data.

You may think of a protocol in the application layer on one device as talking directly to the same protocol in the application layer on the receiving device. However, the path for this "direct" communication is actually provided by the transport layer.

HTTP (HyperText Transfer Protocol) is a request-response protocol for transferring web pages across the internet. It is based on the client-server model. HTTP has two main methods, GET and POST:

• GET is for retrieving data from a server.
• POST is for putting data into a server’s database.

FTP (File Transfer Protocol) is a protocol for transferring files in a client-server model. An FTP server hosts files, and clients can upload, download or remove files if they have permission.

File transfer can be anonymous or non-anonymous (protected), depending on whether you need to identify yourself before the download. A protected site needs a username and password. An anonymous site does not.

• SMTP (Simple Mail Transfer Protocol) is for sending mail to a mail server.
• POP3 (Post Office Protocol version 3) is for receiving mail from a mail server.
• IMAP (Internet Message Access Protocol) is also used to receive mail. Unlike POP3, it keeps the mail on the server, so it stays in sync across all your devices.

It is common for mail to be sent and received using other protocols, such as HTTP. SMTP is still used for moving mail between mail servers.

The transport layer is the second layer of the protocol stack, counting from the top: it sits just below the application layer. Two protocols are used at the transport layer: Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).

The transport layer accepts "messages" from the application layer as application data. It does not care whether this data is part of setting up the connection, actual data to be transferred, or some other control message. It just sees a block of data to be carried to the transport layer at the receiving end.

A port is a number given to every application or process. In communication, data is sent from a sending device to a destination device (the receiver). If the destination device has three applications running, the port number helps to identify the application, or process, that needs the data that has arrived.

• A computer has 65 536 ports in total.
• The port numbers 0 to 1023 are reserved for system use.

When a request reaches the server, the source and destination ports are swapped. For example, if a web browser sends a message to a web server on port 80 (destination) from port 3032 (source), the reply comes back from the web server on port 80 (source) and is sent to the web browser on port 3032 (destination).

This matters because you may have many sessions open to the same resource, for example several tabs in a browser showing different pages of the same website. The source port is generated at random, and it becomes the destination port of the reply, so it identifies exactly which process made the original request.

TCP is often called an "end-to-end" protocol. From the point of view of the application (client or server), using TCP to handle communication is like having a simple serial cable running between the two devices. All the complex detail of how this works is hidden (abstracted) in the lower layers.

A TCP end point is a combination of an IP address and a port number. This combination is called a socket. The socket lasts for the whole of the connection.

A server has to handle many sessions at the same time, as it communicates with many clients. Each session is identified by its own socket. Because each client uses a unique IP address and a 16-bit port number, the server can keep track of up to 65 534 connections per host.

A TCP connection between two sockets is full duplex: the client and the server can both send data at the same time. For any one transmission, one socket is the source and the other is the destination.

TCP (Transmission Control Protocol). Many internet applications, such as web browsing and file transfer, need all of the data. For some activities, such as online banking, there is no room for error. TCP makes sure none of the data is lost.

It does this by keeping track of every segment it makes and giving each one a sequence number. The sequence numbers are added to the segment header. At the receiving end, TCP keeps a record of the sequence numbers that arrive and acknowledges each segment. If the sender does not receive an acknowledgement, the segment is sent again.

The delays caused by waiting for segments to be sent again mean that TCP is not suited to live, real-time uses such as video calls, where late data is of no use.

UDP (User Datagram Protocol). If the speed of data transfer matters most, and the application can put up with losing some data, then UDP, the "unreliable" protocol, can be used. Services that use UDP include real-time applications such as FaceTime and WhatsApp calls, and live media streaming services. Losing some data may cause a blip in the audio or a missing frame in a video, but these losses are acceptable in exchange for fast data transfer.

The internet layer (sometimes called the network layer) receives segments or datagrams from the transport layer. Like the transport layer, it does not care whether they are data or control messages. It only knows that it must wrap them in a suitable package so that they can be routed across the internet to their final destination.

The data-link layer puts data into frames so that it is ready to be sent in the physical layer. It adds both a header and a trailer to the data coming from the network layer.

When data is sent across a network, frames carry the data from one system to the next. This is called hop-by-hop forwarding. When data passes through many systems on the way, each one receives a frame, removes the old frame header and trailer, wraps the data in a new header and trailer for the next hop, and forwards it to the next system.

The BitTorrent protocol is a way of sharing files using a decentralised peer-to-peer model, instead of the traditional, more centralised, client-server model.

In the client-server model, clients download files from a central server. With BitTorrent, a swarm of peers upload to and download from each other. This stops any single device being overwhelmed, because every device shares with only a few others.

To use BitTorrent, a peer loads a .torrent file into a BitTorrent client, which connects it to a tracker. The tracker is a device that keeps a list of the peers connected to it, and tracks which peers have which chunks of which files. The tracker does not store the files; it only keeps track of them. Every so often, peers update the tracker, so that they are not sent chunks they already have.

A peer that has all of the chunks of a file is a seed. Seeds send chunks of the file to peers that do not have the whole file. Once a peer has every chunk, it becomes a new seed and can send chunks to other peers. To keep downloads fast, rarer chunks are given priority.