A Packet’s Tale

Concerning Packets

In the age before cloud, packets were simple things: humble frames of data, born in the NIC of one machine and destined for another, ferried across copper and glass by switches that neither questioned nor cared. But as the world grew more complex, so too did the paths they walked. New kingdoms arose upon the hypervisor, realms of distributed fabric, where routing was no longer the province of great iron lords in the data centre aisle, but was woven into the very kernel of every host. Two orders kept the peace in these realms: the Distributed Routers, who received each packet at the edge of its subnet and decided its fate; and the Distributed Transit Gateways, who carried the chosen onwards, northward and beyond. Neither Edge nor Core, but something in between, quiet, efficient, and everywhere at once.

It is of one such packet that this tale is told. A small packet, as packets go. Unremarkable in its headers. But it had business that required a journey: there and back again.

An Unexpected Redirect

The packet was born in a virtual machine in the western reaches of a VPC subnet, a modest workload doing modest things. It knew its destination: an address beyond the fabric, in the great External Network that lay north of the Transit Gateway. At the edge of its subnet, the Distributed Router received it without ceremony. It was the Router’s duty to decide: to pass it onward to the Transit Gateway, or to send it first on a longer errand.

Now, the Distributed Router was a careful steward. It could forward with great speed and handle flows of east and west without breaking stride. But more than that, it was the keeper of a solemn duty: to inspect each packet’s headers before deciding its path. As it turned this packet over in its inspection (examining the L3 header, reading the L4 fields beneath), it paused.

‘This one needs NAT.’

A stateful service. A rewriting of addresses. The kind of work that requires memory, and state, and a steady hand. Not the work of a Router, however capable. The Transit Gateway must wait.

 

The Road to the VNA

And so the Distributed Router made its decision. The packet was redirected (not cast aside, but sent with purpose) to a place both within the fabric and beside it; woven of the same cloth, yet set off the main road, as a quiet hall stands within a great city but away from the thoroughfare. A Virtual Network Appliance: lighter than an Edge Cluster, built for precisely this kind of errand. It sat in an Active/Standby pair, vigilant, its Standby companion ever watchful should the Active falter.

The packet arrived. The VNA received it without fuss, consulted its translation tables, and set to work. The source address in the L3 header was rewritten. The port in the L4 field was changed. What had entered as one thing departed as another (same payload, new identity), ready now to walk in the External Network without betraying its true origin.

There and Back Again

The packet did not linger. Its business at the VNA was done. It turned back the way it came, back to the Distributed Router, which had been waiting with the patience of one who knows its part. The Router examined the rewritten headers, found them good, and passed the packet at last to the Distributed Transit Gateway. The Gateway forwarded it northward, out through the fabric, and on to its destination beyond.

No Edge Cluster had been summoned. No great iron lord had stirred. A small packet had gone there, been changed in the way it needed to be changed, and come back again; and the fabric had never missed a beat.

This was written by my colleague Alasdair Carnie and I thought it was brilliant, so I am passing it along.