Traffic Engineering

Traffic engineering is the practice of deliberately influencing which path packets traverse through a network, independent of failure recovery or name‑based redirection. It matters when a network has multiple physical or logical paths between points and you need to steer flow for reasons of utilization, latency, cost, or policy—rather than simply waiting for a failure to trigger reroute. Unlike redundancy, which restores service after a break, traffic engineering proactively assigns preferences so that “normal” operation already uses the desired path. It is also distinct from DNS‑based routing, which redirects before the TCP session establishes, and from application‑layer gateways that make routing decisions at layer 7. At its core, traffic engineering operates at layers 3 and below, manipulating routing metrics, BGP attributes, or packet‑by‑packet policies to shape the data plane.

Steer a source across a chosen uplink

When a host or service must egress through a specific upstream link—perhaps because of SLAs, peering agreements, or cost differences—policy‑based routing (PBR) provides packet‑by‑packet control without requiring the host’s own routing table to change.

On a Linux host or router, add a routing table for the uplink and a rule that selects it based on source address or packet marks:

# Create a dedicated routing table
echo "200 uplink1" >> /etc/iproute2/user_routes

# Add a default route into that table via the desired interface
ip route add default via 10.1.1.1 dev eth1 table uplink1

# Select that table for traffic from a specific subnet
ip rule add from 192.168.10.0/24 table uplink1

Verify the rule and table:

ip rule show
ip route show table uplink1

When a packet originates from 192.168.10.x, the kernel consults the rule list, matches the source, and installs the default via eth1 from the uplink1 table. Traffic from other sources continues to use the main table or other defined tables. If the rule exists but the target route is absent, the packet is silently dropped—no ICMP notification is sent, and monitoring may show normal inbound bytes on the primary interface while the intended uplink carries nothing.

To mark packets explicitly (useful when source‑address based rules are insufficient), use tc (traffic control) to set a fwmark, then rule on that mark:

# Mark outbound packets from a process
iptables -t mangle -A OUTPUT -p tcp --dport 443 -j MARK --set-mark 1

# Rule to use the marked packet’s table
ip rule add fwmark 1 table uplink1

A common silent failure occurs when the mark is set but the kernel’s policy database has not been re‑loaded after a reload, or when multiple rules match and the first one in the list (by priority) wins, starving the intended table.

Influence inbound traffic from BGP peers

When an autonomous system receives traffic from multiple upstreams, BGP path‑selection attributes can be tuned so that peers preferentially send certain destinations over specific links. The two most frequently confused attributes are MED (Multi‑Exit Discriminator) and local‑preference; they operate at different stages of the decision process.

• local‑preference is sent and evaluated *within* your AS. A higher value is preferred when choosing among outbound paths. It does not leave the AS.
• MED is sent *to* neighboring ASes and influences how those neighbors prefer one of your exit points over another. A lower MED is preferred by the receiving peer.

No vendor‑specific configuration key is shown here; the mechanics are standard across BGP speakers. A typical route‑map logic, expressed in abstract terms, looks like this:

if destination in {critical‑services} then
    set local-preference 200
else if destination in {cost‑sensitive} then
    set metric 10
endif

To view the current BGP best‑path selection and the attributes that drove it, many operators query a looking‑glass or use a generic BGP client:

# Example using bird (open‑source BGP daemon) control interface
birdc show route <prefix>

The output will list the best path, its originator, and the values of locpref, med, as-path, and other factors. If inbound traffic from a peer consistently enters via the wrong link, check whether the peer is sending a MED that your policy expects, or whether your local‑preference is being set correctly for the relevant address prefixes. A frequent edge case: a peer advertises the same prefix with different MEDs on two links, but your router’s always‑compare‑med flag is not enabled, so the MED is ignored and the decision falls back to AS‑path length or router ID—often an unexpected path.

Engineer MPLS‑TE tunnels for deterministic paths

In MPLS‑core networks, traffic engineering is often achieved by label‑switched paths whose routes are computed and resource‑reserved at tunnel‑setup time, diverging from the IGP’s shortest‑path calculation. A TE tunnel is not a “static route” in the IP sense; it is a signaling‑driven LSP that can be constrained by bandwidth, affinity, or explicit sub‑paths.

Key operations, expressed generically:

• Create a tunnel that reserves bandwidth from the headend to the tailend, independent of IGP paths.
• Assign traffic to the tunnel via a penultimate‑penultimate PHP (penultimate hop popping) label, or by matching the packet’s DSCP/exp field and imposing the tunnel label.
• Verify that the tunnel is up and carrying traffic:

# On a Juniper or Cisco device (output format varies, mechanism is generic)
show mpls lsp extensive | match <tunnel-name>

• Inspect reservation:

show mpls traffic-eng tunnels

If a tunnel is administratively up but not carrying traffic, the most common silent failure is that the tunnel’s path cannot be resolved because the required link’s bandwidth has been exhausted and the autoroute feature is not enabled, so the IGP still forwards packets via the shortest‑path LSP. Another edge case: explicit paths that include a link that later becomes congested; the tunnel stays up (LSP up) but packets are dropped because the signaled bandwidth is no longer available—no teardown occurs, and traffic appears to “vanish” from the tunnel’s egress.

Recover from a degraded path without full reconvergence

Traffic engineering also encompasses techniques that allow a network to shift flow when a path degrades but before the IGP or BGP fully reconverges. Two widely used mechanisms are:

1. Backup paths in ip route: a secondary next‑hop that the kernel selects only when the primary path’s metric exceeds a threshold or the next‑hop becomes unreachable.
2. BGP PIC (Prefix‑Independent Convergence): a route‑refresh‑style mechanism where a backup path is already installed in the RIB, and upon a primary path failure, traffic switches in under a second without waiting for the full BGP table merge.

Linux ip route backup syntax:

ip route add 10.0.0.0/8 via 172.16.0.1 dev eth0 \
    weight 10 \
    backup via 172.16.0.2 dev eth1

Here, weight influences the probability of selection when both next‑hops are up (higher weight = more traffic), while backup ensures the second next‑hop is used only if the primary fails. Verification:

ip route show 10.0.0.0/8

A sharp edge: if the primary next‑hop becomes reachable but the link’s latency spikes above a usable threshold, the kernel does not automatically switch to the backup; the backup only activates on complete next‑hop unreachability. Operators often pair this with health‑check–driven route‑withdrawal or use BGP PIC for faster switchover.

Another pattern: IGP metric manipulation. Raising the cost of a link in OSPF or IS‑IS makes the router prefer an alternate path for all traffic that uses that IGP area. This is a “soft” engineering knob—change the metric, wait for LSA flood, and traffic shifts. The sharp edge here is that metric changes propagate to all destinations using that link, not just the targeted flow, and a mis‑step can blackhole traffic if the new metric creates a routing loop or a stub network that no longer appears in the shortest‑path tree.

Sharp edges you only learn after it breaks

• Metric interoperability: Linux ip route metrics, OSPF cost, BGP MED, and MPLS TE bandwidth reservations are all called “metric” but are not interchangeable. A change in one domain does not affect the others, and operators have been surprised when OSPF cost adjustments had no impact on BGP‑selected paths because the decision had already been made at the UPDATE phase.
• Rule ordering blind spots: ip rule entries are evaluated in order of priority (lower number first). Adding a new rule without removing or re‑prioritizing existing ones can silently redirect traffic that was intended for another policy. The kernel never warns about overlapping intentions.
• PBR and NAT interaction: When policy‑based routing redirects a packet into a custom routing table, NAT may not be performed because the packet’s source address no longer matches the interface’s primary address range. The packet is forwarded, but the translation step is skipped, and the remote endpoint sees an unexpected source IP.
• MPLS TE tunnel teardown lag: A tunnel may be signaled up while the underlying link’s reserved bandwidth has been reclaimed by other TE tunnels. The control plane reports “up”, the data plane drops packets, and monitoring shows “traffic engineering efficiency” dropping without any explicit error flag.
• BGP MED vs local‑pref confusion: In multi‑homed designs, a common production outage occurs when an operator raises local‑preference to prefer one exit, but the upstream peer ignores it because the MED advertised on that exit is higher (less preferred). The result is that traffic still egresses via the lower‑preference path, and the operator spends hours verifying route‑maps before realizing the peer’s decision logic is MED‑driven, not local‑pref‑driven.

Traffic engineering is not a set‑and‑forget configuration; it is a continuous practice of measuring path characteristics, adjusting influencing attributes, and validating that the data plane respects the intended policy. The tools—ip, route, tc, BGP speakers, MPLS signaling—are standard, but the interactions between them are fragile. A disciplined approach of “change one thing, verify the full path, then proceed” is the only reliable way to keep traffic engineering delivering its intended benefit without introducing silent black‑holes or performance cliffs.

This page was rewritten on 18 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.