Visualising BGP-LS Tables (Python, GoBGP, gRPC, NetworkX)

Recent work on refactoring my existing solution to graphing LSDBs with Python prompted a discussion with my good friend Mark (“Solar”) on how we could make the solution vendor-agnostic, and dynamic.

Owing to my recent and historic lab-work with XTC[8] and WAE (via dCloud)[1], and Mark’s professional interaction with Northstar[3], we concurred that the obvious solution towards collecting the LSDB in a vendor-agnostic manner is to make use of the BGP-LS address family, used by “offline path calculation” for RSVP-TE LSPs (i.e. Northstar).

BGP-LS

BGP-LS, in its current form, originated (we believe) in RFC7752 [7] – a proposed method of translating LSDB items (LSA or LSPs) from the Link-State protocol into MP-BGP NLRI, for transport via a new AFI/SAFI pair. That is, communicating the LSDB over BGP.

It’s most apparent use-case is what Osborne’s book Traffic engineering with MPLS[4] describes as “Offline Path Calculation” – a mechanism towards solving the packing problem by maintaining an out-of-band oracle of a Link-state topology, link/path utilisation, and MPLS LSP state. This provides a reduction in state for the whole network, delegated to a central entity outside the forwarding path, and allows for a birdseye view of your RSVP-TE LSPs as they operate in the field. (Centralised command-and-control for MPLS-TE).

The topology information required to build and maintain such a tool, before any LSPs can be signalled or forwarding-state can be collated, falls to the Link-State protocol in use. Recall that Link-State protocols maintain full-views of the network topology, with periodic flooding as their mechanism for update. This “full view”, termed the Link-State Database (LSDB), is the eyes for any offline path calculation element.

If the LSDB are the eyes, BGP-LS is the nerves and synapses connecting them to the brain: an SDN WAN controller.

Task

To begin, we set our goals as the following

  1. Host a BGP-speaking tool that is ready/able to peer BGP-LS, and permits interaction with python over the network
  2. Stand-up a lab running ISIS, with an edge node peering with BGP-LS to our BGP-speaking tool.
  3. Write Python that connects to the BGP-speaking tool and pulls the BGP-LS table.
  4. Graph the topology.

For our BGP-speaking tool, we opted for GoBGP

GoBGP is an open source BGP implementation designed from scratch for modern environment and implemented in a modern programming language, the Go Programming Language.

https://github.com/osrg/gobgp

Deliverable

A link to the Github hosting the solution can be found here.

bgp-ls-vis is a PoC collection of scripts and modules that perform the following:

  1. Connects via gRPC to a running GoBGP instance and submits a query for the contents of the BGP-LS table
    • or, loads from file a cached BGP-LS table
  2. Filters the returned structured data for only those values which are useful and required
  3. Builds a NetworkX graph object representative of the NLRI extracted
  4. Draws visual representation of said NetworkX graph object to screen or file

In addition to the above primary tasks, the deliverable also permits loading YAML dumps of the BGP-LS table pulled from GoBGP. This allows for drawing topology diagrams using the NLRI of a cached BGP-LS table, forgoing a gRPC connection to the GoBGP instance. This will also allow us to create unit tests, where a gRPC connection would not be viable.

Because the design is modular, the graphing solutions are separated and disparate to the BGP-LS and LSDB gathering effort. Several graphing libraries can be used, as we prep a topology in NetworkX before graphing it. For example, we present graphing approaches using Pyplot, Graphviz, and Dash (A web-framework).

An 18-router IS-IS topology is shown below. Note the following:

  • There is support for Pseudonode segments, demonstrated by the blue node below, for a broadcast segment between P13, 14 and 18.
  • TLV 137 (Hostname) information is extracted where present and used to name the nodes by hostname. Where not present, the IGP RID is used instead.
  • Note the bidirectionality of links. We use a Directed-Multigraph to achieve this.

Part 1 – Initial Design and Testing Topology

The design calls for 3 components:

  1. The network itself – a Link-State routing protocol, including an edge-node peering BGP-LS
  2. An instance of GoBGP, running BGP-LS, peered to the network
  3. The script that we author, which connects via gRPC to the GoBGP instance to gather BGP-LS NLRI
High-level overview of the components involved in our solution

We begin with a lab topology in GNS3 that includes the following:

  • 18 Cisco devices: 17x IOS-Classic images (7200), 1x CSR1000v
  • An ubuntu VM, with two ports: South to the topology, north to the real-word, via …
  • A cloud node, bridging the VM from the lab to the real-world, over which gRPC connections will eventually move.
Lab topology in GNS3

Some items to note about the above topology

  • We initially attempted this solution using OSPF, but quickly ran on an issue where GoBGP was unable to parse the BGP-LS NLRI sent from the CSR1000v node (R2). An issue presented with the IGP RID bit format in the payload. As such, we switched to IS-IS.
  • With a BGP-LS session established to GoBGP, we attempted to verify it, wherein GoBGP threw a null-reference error. Research on the GoBGP bug-tracker revealed that the BGP-LS AFI is implemented for API use only, so we had no way of viewing the NLRI in the table from the bash shell. However, we were able to successfully correlate the count of NLRI transmit at the CSR1000v, and received at the GoBGP speaker – the values were identical, telling us all NLRI was moving successfully.
  • Every link is of network type Point-to-Point, except the segment between routers 13, 14 and 18, which is a broadcast type network. This is to enable handling of pseudonodes (Read “DR/BDR” in OSPF nomencalture).

Finally, owing to Marks experience and preference towards Junos, he provided comparable topologies using vMX devices, towards his own GoBGP server, and dumped BGP-LS NLRI for true vendor-agnostic testing.

Config Snippets

The GoBGP configuration file is defined by the following YAML, called by: sudo -E ./gobgpd -f gobgpd.conf -t yaml

global:
  config:
    as: 65001
    router-id: 10.2.9.9
neighbors:
- config:
    neighbor-address: 10.2.9.2
    peer-as: 65001
  transport:
    config:
      local-address: 10.2.9.9
  afi-safis:
  - config:
      afi-safi-name: ls

IOS-XE on CSR1000v permits distribution (not redistribution) of Link-State routing information to BGP-LS through the steps detailed in Cisco documentation.[6]

Configuring IS-IS With Border Gateway Protocol Link-State
Perform the following steps to configure IS-IS with BGP-LS:

Enable the IS-IS routing protocol and enter router configuration mode.

Device(config-router)# router isis
Distribute BGP link-state.

Device(config-router)# distribute link-state 
Configuring BGP
Perform the following steps to configure BGP with BGP-LS:

Enable the BGP routing protocol and enter router configuration mode.

Device(config-if)# router bgp 100
Configure the address-family link-state.

Device(config-router)# address-family link-state link-state

Confirmation that the BGP session between the CSR1000v and GoBGP is UP is given by both platforms, and we observe the sent NLRI count vs the received NLRI count.

{"level":"info","msg":"gobgpd started","time":"2021-01-09T11:51:04-05:00"}
{"Topic":"Config","level":"info","msg":"Finished reading the config file","time":"2021-01-09T11:51:04-05:00"}
{"level":"info","msg":"Peer 10.2.9.2 is added","time":"2021-01-09T11:51:04-05:00"}
{"Topic":"Peer","level":"info","msg":"Add a peer configuration for:10.2.9.2","time":"2021-01-09T11:51:04-05:00"}
{"Key":"10.2.9.2","State":"BGP_FSM_OPENCONFIRM","Topic":"Peer","level":"info","msg":"Peer Up","time":"2021-01-09T11:51:06-05:00"}
osboxes@osboxes:~$ ./gobgp nei
Peer        AS  Up/Down State       |#Received  Accepted
10.2.9.2 65001 00:00:25 Establ      |       41        41
P2#show bgp link-state link-state summary | i bestpath
41/41 BGP path/bestpath attribute entries using 12384 bytes of memory

Part 2 – gRPC connection to the GoBGP instance, and pulling BGP-LS NLRI

The unfortunate nature of gRPC with python requires that you prepare interfacing scripts before making a connection. The procedure is detailed on the GoBGP readme, and thankfully is acheived by one command:

Generating Interface
You need to generate the server and client interface from GoBGP proto files at first.


$ python -m grpc_tools.protoc -I./ --python_out=. --grpc_python_out=. *.proto

$ ls *.py
attribute_pb2.py  attribute_pb2_grpc.py  capability_pb2.py  capability_pb2_grpc.py  gobgp_pb2.py  gobgp_pb2_grpc.py

From this, the 6 files above are generated. We place this in the same directory as a testing script, __init__ for our module, and employ the required gRPC calls to establish a connection.

from google.protobuf.json_format import MessageToDict

# RPC & GoBGP imports
import grpc
import gobgp_pb2 as gobgp
import gobgp_pb2_grpc
import attribute_pb2

channel = grpc.insecure_channel(f"172.20.10.2:50051")
stub = gobgp_pb2_grpc.GobgpApiStub(channel)

request = gobgp.ListPathRequest(
  table_type=gobgp.LOCAL,
  name="",
  family=gobgp.Family(afi=gobgp.Family.AFI_LS, safi=gobgp.Family.SAFI_LS),
  prefixes=None,
  sort_type=True,
)

response = self.stub.ListPath(request)
table = [MessageToDict(nlri) for nlri in response]

The above code should facilitate establishing a gRPC connection to your GoBGP instance and pulling the full BGP-LS table, finally formatting it to a dictionary of lists-and-dicts; the familiar format for those of us that have made RESTful queries elsewhere.

An example BGP-LS table dump for the 18-node topology can be seen here. (tests/bgp-ls_table_dumps/18-node-isis-w-bcast-segment.yaml)


With the BGP-LS table acquired, the final step is parsing it to a format that is more concise and relevant to our use case. From the structure described when presented as YAML, you quickly identify single-item lists, objects types in dedicated “type” keys, and a hierarchy of TLVs and empty fields that we simply do not need.

As such, we define a method get_lsdb which parses a raw BGP-LS table into a data-structure far simpler to navigate. This is our LSDB, and in it are 3 types of LSA: Node, Link, and Prefix – correlating to defined NLRI types in the BGP-LS RFC.

- linkDescriptor:
    interfaceAddrIpv4: 10.3.4.3
    neighborAddrIpv4: 10.3.4.4
  localNode:
    asn: 65001
    igpRouterId: 0020.9000.0003
  lsattribute:
    link:
      bandwidth: 125000000.0
      igpMetric: 10
      localRouterId: 2.90.0.3
      remoteRouterId: 2.90.0.4
      reservableBandwidth: 12500.0
      unreservedBandwidth:
      - 12500.0
      - 12500.0
      - 12500.0
      - 12500.0
      - 12500.0
      - 12500.0
      - 12500.0
      - 12500.0
    node:
      localRouterId: 2.90.0.3
    prefix: {}
  remoteNode:
    asn: 65001
    igpRouterId: 0020.9000.0004
  type: Link

- localNode:
    asn: 65001
    igpRouterId: 0020.9000.0008
  lsattribute:
    link:
      localRouterId: 2.90.0.8
    node:
      isisArea: SRI0
      localRouterId: 2.90.0.8
      name: P8
    prefix: {}
  type: Node

- localNode:
    asn: 65001
    igpRouterId: 0020.9000.0001
  prefixDescriptor:
    ipReachability:
    - 2.90.0.1/32
    ospfRouteType: LsOspfRouteType(0)
  type: Prefix

Part 3 – Drawing the LSDB with Graphing Tools (NetworkX)

The final step of our effort is to take the above-descrived LSDB format and construct in-memory a topology representative of the BGP-LS NLRI collected.

We acheive this using NetworkX.

NetworkX is a Python package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.

https://networkx.org/

graphing.build_nx_from_lsdb(lsdb: list) -> nx.MultiDiGraph – a method that translates our LSDB to a NetworkX graph object. It builds on previous work demonstrated in the Link-State visualiser that inspired our work, and the work that came before that: https://github.com/theclam/ospfcli2dot

NetworkX Graph objects are comprised of NetworkX Node and Edge objects. A graph object can be one of many types of graph, the most appropriate for a Link-State topology being a Weighted-Directed-Multi-graph

  • Multi-Graph, allowing several edges between the same two vertices
  • Directed, allowing an edge to have a property of unidirection
  • Weighted, allowing Link-State cost to represent a metric for traversing a given edge

Additionally, NetworkX node and edge objects permit the inclusion of arbitrary data, which permits us to collate the tertiary attributes included in the BGP-LS NLRI as a property of said graph object. We use this to signifify the following, incomplete list of attributes

  • The node / edge connects a Pseudonode
  • The TED bandwidth constraints defined for an edge
  • TLV137 – Node hostname
  • Arbitrary well-known keys for drawing NetworkX graphs, such as color, and line-weight

The quickest and easiest drawing library we found was Pyplot. It is quick and minimal, although not particularly pretty.

graphing.draw_pyplot_graph for BGP-LS table dump junos_bgpls_nopsn.yml draws the above
graphing.draw_pyplot_graph for BGP-LS table dump 18-node-isis-w-bcast-segment.yaml draws the above

Future

“OpenStar” was a fun potential title that probably describes our long-term goals for this project or any other works it might motivate. WAE/Crosswork and Northstar have only 1 real open-source competitor, OpenDaylight, which whilst supporting BGP-LS, IPFIX and PCEP, doesn’t provide a front-end.

What if we could provide a front-end for an ODL instance to achieve analogous solutions to those solved by Northstar, or we could write our own IPFIX and PCEP tools for the existing solution to achieve offline path calculation? What a learning exercise it would be to build our own open-source Northstar.

However, more immediate work is required before such lofty goals.

  • Mark’s work on a web-based frontend using Dash is making great progress, and we continue to work together to get something that would be engineer-facing and pretty.
  • We must make the solution dynamic and intelligent, so that it pulls recent BGP-LS information, and compares to a cached or trusted copy of the baseline topology, towards detecting and signifying faults (I.e. a Link or node dropping).
  • Anything else we can feasibly pull from a BGP speaking client to allow us to draw all sorts of things atop the topology, such as charting the path of a signalled LSP.

These are all ideas we are considering, but work is by no means complete. In it’s current state, this is a proof-of-concept solution to graphing the BGP-LS table, and nothing more.


If you would like to talk more about the solution we’ve prepared, I invite you to contact me on LinkedIn, or to join the Networking discord at https://discord.me/networking

References and Addendum

  1. Cisco dCloud. 2021. Cisco WAN Automation Engine 7.1.3 With NSO, SR, SR-PCE, XTC, And Crosswork Situation Manager V2. [online] Available at: <https://dcloud2-lon.cisco.com/content/demo/402876?returnPathTitleKey=content-view&gt; [Accessed 13 January 2021].
  2. FUJITA, T., 2021. Gobgp. [online] Osrg.github.io. Available at: <https://osrg.github.io/gobgp/&gt; [Accessed 13 January 2021].
  3. Juniper Networks. 2021. Northstar WAN SDN Network Controller – Juniper Networks. [online] Available at: <https://www.juniper.net/uk/en/products-services/sdn/northstar-network-controller/&gt; [Accessed 13 January 2021].
  4. Osborne, E. and Simha, A., 2003. Traffic Engineering With MPLS. Indianapolis, IN: Cisco, pp.426-431.
  5. Singh, D., 2021. Yet Another New BGP NLRI: BGP-LS – Packet Pushers. [online] Packetpushers.net. Available at: <https://packetpushers.net/yet-another-new-bgp-nlri-bgp-ls/&gt; [Accessed 13 January 2021].
  6. Support, P., Routers, C. and Guides, C., 2021. IP Routing: BGP Configuration Guide – Border Gateway Protocol Link-State [Cisco ASR 1000 Series Aggregation Services Routers]. [online] Cisco. Available at: <https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-16/irg-xe-16-book/bgp-ls.html#concept_FE324DAD231D4288AE9DB0A3B6B45731&gt; [Accessed 13 January 2021].
  7. Tools.ietf.org. 2021. RFC 7752 – North-Bound Distribution Of Link-State And Traffic Engineering (TE) Information Using BGP. [online] Available at: <https://tools.ietf.org/html/rfc7752&gt; [Accessed 13 January 2021].
  8. Trate, F., 2021. Add A Dollop Of XTC To Your Network Infrastructure – Cisco Blogs. [online] Cisco Blogs. Available at: <https://blogs.cisco.com/sp/add-a-dollop-of-xtc-to-your-network-infrastructure&gt; [Accessed 13 January 2021].

Thanks to contributor Mark “Solar” Findlay – https://markfindlay.net/

One thought on “Visualising BGP-LS Tables (Python, GoBGP, gRPC, NetworkX)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s