All we want for X-mas 🎄: Hyper-scale targeted LDP signaled MACsec 🔐 protected Traffic Engineered epipe ∿ services using Segment Routing over ISIS ☥

4 min readDec 29, 2021

Featuring Netsim-Tools 1.0.6+ by Ivan Pepelnjak

Traffic Engineering in SR-ISIS networks (from Derek’s post)

As the dust slowly settles over 2021, some of you may be wondering if that is it for this year. As it turns out, it is not — there is more! Ivan just released version 1.0.6 of Netsim-Tools, and it’s a doozy.

Extensible Model-Driven Architecture: Truly 😮pen™ collaboration

What is so great about Netsim-Tools and this latest release in particular, is that it is fully customizable by design. The new plugin mechanism allows anyone to add whatever “nerd knobs” they need for their particular use case. Just like SR Linux, Netsim-Tools is truly open — which makes it a great tool to build upon and collaborate with.

Evolutionary paths — some platforms are more open than others

[…] The animal species, in which individual struggle has been reduced to its narrowest limits, and the practice of mutual aid has attained the greatest development, are invariably the most numerous, the most prosperous, and the most open to further progress — Peter Kropotkin on “survival of the fittest”

Last week I already talked about Segment Routing (SR) support for 7250 IXR devices running the new SR Linux 21.11.1 release. While we could use that to implement the SR-ISIS part of the scenario suggested by Derek Cheung (back in 2020) — i.e. R2 and R3 in the diagram — R1 and R4 need to use SR OS, and given my curiosity about SR Traffic Engineering (a.k.a. decorating the network with tunnels all over the place — exciting new territory…) I decided to go all-in on SR OS this time.

So here’s what we’ll be doing today:

Recreate Derek’s SR-ISIS MPLS with Traffic Engineering use case, using Netsim-Tools (extended/enhanced with various “nerd knobs” as needed) and Containerlab
Explore what it would take to add MACsec encryption to the L2 service

Github project can be found here; requires this pending PR (and perhaps a little patience…)

ePipe services — adding custom L2 networking

Netsim-Tools provides the core functionality to model routed network topologies, but prefers to leave (vendor-specific) details to custom plug-ins. For this exercise I’ve created a small ‘epipe’ plugin module which resolves a custom link attribute containing a node name to a nexthop loopback IP address, for use in a custom Jinja2 template.

On the network side, this introduces the need for L2-only ports — i.e. ports without any IP address assigned. To support this, I modified Netsim-Tools to allow for “ipv4: False” (and since I was in the neighborhood, I also added index-based addressing where users can specify an offset relative to a subnet prefix)

Seamless hyper-scale security-as-a-service at layer 2

As an aside: The reason we (i.e. including you) need this type of L2 connectivity service is also articulated here. Security is a key requirement and investment driver for enterprises, and being able to add it to an existing network service seamlessly — meaning without customers having to make any changes — is essential. Doing that at scale means automation; in other words, what I’m showing here is how we can model and automate security services at hyper-scale, in a form that you can experiment with in your lab.

BFD module

Bidirectional Forwarding Detection (BFD) as defined in RFC5880 is a staple of rapid fault detection and associated failover mechanisms. In this use case it is enabled for ISIS interfaces, using parameters defined in the RFC. One could do the same for OSPF and BGP — but we’ll leave those for another day. The same goes for “seamless bfd” (RFC7880) extensions.

MACsec on 7x50 SROS

Nokia SR OS has long supported Media Access Control security (MACsec, formally IEEE 802.1AE), a security feature supported in hardware on certain MDAs. I could not find a simple table to tell me which MDAs support it, but the vSIM installation guide lists the ‘maxp10–10/1gb-msec-sfp+’ MDA for sr-a4/a8 — so I guess “msec” means MACsec and the sr-a4 is our guy.

Except…Containerlab(Vrnetlab) didn’t support that particular model. Not to worry — we can add that — and so we are good to go.

Or so I thought

As it turns out, MACsec has more of a hop-by-hop architecture, and doesn’t “just” work across a multi-hop MPLS network; the MKA session doesn’t come up. There is a ‘eapol-destination-address’ (MAC address) under port/ethernet/dot1x/macsec which looks promising — but this would require R1 and R4 to have a MAC address to target, which they currently don’t. Maybe a PBB VPLS could work?

Besides that, the MTUs will need some tweaking: MACsec adds 32 bytes to each packet.

Let’s figure it out together — suggestions appreciated!