How Communication Networks Move Data: Layering, Congestion, and Routing

Moving data across a network is not one operation. An application creates bytes; transport identifies endpoints and may provide reliability; IP selects a path across heterogeneous links; each link carries a frame to the next hop. The design works because every layer exposes a service while hiding most of its implementation.

The most useful way to study this system is end to end: follow one flow, identify where state lives, and separate delivery, reliability, congestion control, forwarding, and routing. Treating these functions as interchangeable is the source of many incorrect network explanations.

Layering is a contract

A protocol defines the format, meaning, and ordering of messages exchanged by peer entities. A service describes what one layer offers the layer above; an interface describes how that service is accessed. These concepts are related but not identical.

When an application sends data, each layer adds control information needed by its peer. The receiver removes those headers in reverse order. The apparent peer-to-peer conversation at a layer is implemented by actual transmission through all lower layers. This encapsulation lets TCP operate over Ethernet, Wi-Fi, cellular links, or tunnels without knowing each medium’s signaling details.

Layering is not absolute isolation. Path maximum transmission unit, explicit congestion notification, hardware offload, and application-aware transport all cross traditional boundaries for performance. The architectural test is whether such coupling preserves a clear contract rather than whether it follows a diagram perfectly.

Packet switching creates queues

Packet networks statistically multiplex many flows over shared links. Unlike a dedicated circuit, a flow does not reserve the entire path. Utilization improves, but packets can wait, arrive out of order, or be dropped when buffers and links are overloaded.

An end-to-end delay budget can be decomposed as

d_{\mathrm{end\text{-}to\text{-}end}} =\sum_{k=1}^{K}\left(d_{\mathrm{tx},k}+d_{\mathrm{prop},k}+d_{\mathrm{queue},k}+d_{\mathrm{proc},k}\right).

Transmission delay depends on packet length and link rate; propagation delay depends on distance and signal speed; processing delay covers protocol work; queueing delay depends on competing traffic and is usually the most variable term. A low nominal link latency therefore does not guarantee a low tail latency under load.

Switches and routers need both lookup and scheduling. Ethernet switches learn source MAC addresses and forward frames using a filtering database. IP routers perform longest-prefix matching in a forwarding information base. Queue disciplines such as FIFO, priority scheduling, and weighted fair queueing choose which packet leaves next; that choice directly affects fairness and latency isolation.

MPLS adds a stack of 32-bit label entries and forwards according to labels along a label-switched path. It can simplify traffic engineering, but it does not make routing or queueing disappear: control-plane mechanisms must still establish paths, and each output link remains a shared resource.

IP offers reachability, not reliable delivery

IP provides connectionless, best-effort datagram delivery. A router may drop a packet, and packets may take different paths. Reliability is deliberately left to endpoints or applications so that the network core can remain general.

IPv4 routers may fragment packets when permitted, although path-MTU discovery and avoiding fragmentation are preferable. IPv6 has a fixed 40-byte base header plus optional extension headers; routers do not fragment IPv6 packets, so fragmentation is performed only by the source. ICMP and ICMPv6 report errors and support diagnostics. ICMPv6 also carries neighbor discovery and related IPv6 control functions; it is not merely a renamed version of ICMP for IPv4.

Addressing and routing are separate. An address identifies an interface within a routing architecture; the forwarding table decides the next hop. NAT rewrites addresses or ports at an administrative boundary, which conserves public IPv4 addresses but introduces state and weakens the original end-to-end addressing model. NAT is not a security boundary by itself.

Transport separates application flows

UDP offers message-oriented datagrams with ports and a checksum, but it does not provide retransmission, ordering, or congestion control. An application using UDP must decide which of those functions it needs. TCP offers a reliable ordered byte stream. A connection is identified by endpoint addresses and ports, while the operating system maps that flow to a socket.

TCP reliability combines sequence numbers, cumulative acknowledgments, retransmission timers, and a sliding window. The receiver-advertised window protects the receiver from a sender that is too fast; this is flow control. The congestion window protects the network from excessive in-flight traffic; this is congestion control. The sender is limited by both, so a rough steady-state bound is

\mathrm{throughput}\lesssim\frac{\min(\mathrm{cwnd},\mathrm{rwnd})}{\mathrm{RTT}}.

This expression is a mental model, not a complete TCP performance equation. Loss recovery, delayed acknowledgments, pacing, application stalls, receiver behavior, and the selected congestion-control algorithm all matter.

SCTP provides reliable message delivery, multiple streams, and multihoming within an association. Its cookie mechanism postpones state allocation during association setup and mitigates some resource-exhaustion attacks, but it does not make SCTP immune to denial of service. Protocol security claims should always be tied to a specific threat model.

Congestion control stabilizes shared capacity

Flow control asks whether the receiver can keep up. Congestion control asks whether the network can carry all offered traffic. When aggregate arrival rate exceeds a bottleneck’s service rate, queues grow; once buffers fill, loss and retransmission can amplify the overload.

Classic additive-increase, multiplicative-decrease behavior can be summarized as

W\leftarrow W+\alpha \quad \text{per round trip}, \qquad W\leftarrow \beta W \quad \text{after congestion},\quad 0<\beta<1.

TCP also uses slow start, retransmission timeouts, and loss recovery. Reno is historically important, but it is not a description of every modern TCP. CUBIC, for example, uses a different growth law. At this article’s 2022 date, its then-current specification was Informational RFC 8312, designed to improve scalability on fast, long-distance networks. Explicit congestion notification can signal congestion without requiring a packet drop.

The practical lesson is to inspect the actual endpoint algorithm and path. A timeout can reflect congestion, reordering, wireless loss, or an abrupt route change. Inferring a unique network cause from a transport symptom is unsafe without measurements.

Forwarding is local; routing is distributed

Forwarding is the data-plane action applied to one packet: find a matching entry and send the packet to a next hop. Routing is the control-plane process that creates and updates those entries from topology, reachability, metrics, and policy.

For a graph with link costs $w_e$ , shortest-path routing solves

P^\star=\underset{P\in\mathcal{P}(s,t)}{\operatorname{arg\,min}}\sum_{e\in P}w_e.

Distance-vector protocols exchange distance estimates with neighbors; link-state protocols flood topology information and let each router compute paths. OSPF is a link-state interior gateway protocol. BGP exchanges reachability and path attributes between autonomous systems; its decisions are policy-driven and cannot be reduced to globally shortest paths.

The routing information base may contain several candidate routes learned from connected interfaces, static configuration, and protocols. The selected routes are installed into a forwarding information base optimized for packet lookup. Convergence means that routing state has adapted after a change; transient disagreement can still produce loops, black holes, or reordering.

Diagnose from the correct layer

An effective network investigation follows the packet and tests one boundary at a time:

Resolve the destination name and confirm the address family.
Check the local route and next hop.
Verify link reachability and path MTU.
Measure loss, RTT distribution, and queueing under load.
Inspect transport state, retransmissions, windows, and the active congestion algorithm.
Test application behavior separately from network reachability.

No single metric explains the whole path. Ping does not validate a TCP service; successful TCP setup does not prove acceptable throughput; average latency can hide severe tail delay. Layering becomes useful when it guides evidence collection rather than when it is memorized as a list.