PoC Network Topology Discovery using Slurp’it

The solution to this dilemma is not trivial. Traditionally, we have resorted to running a series of commands on the network, such as Netmiko or Ansible, to gather logs and then process them with text FSM templates, often creating multiple dictionaries and lists to organize the data. While this methodology works, its scalability is questionable.

Slurp’it offers an interesting solution for network planning and data collection. It automates these processes and stores the data in a database that is always available. You get to decide how you want to access and process the information.

However, Slurp’it is not the only tool on the market that performs this task. While there are other options, what distinguishes Slurp’it IMHO is its flexibility. It allows you to define what data to collect and how to store it, an essential feature for adapting to each network’s specific needs. This customization capability is crucial, as each organization has its requirements and data models.

From my experience, I can confidently say that the design of the data model is fundamental. The way we interpret and represent information dictates how we interact with it. For example, the definition of a router may vary among individuals, some see it as a device with interfaces, while others consider interfaces as separate entities. These differences may seem subtle, but they can have a significant impact on how network data is managed and analyzed on a large scale.

    name: routerA,
    interfaces: [{
        name: toRouterB
    }]}

interfaces: [{
    name: toRouterB,
    router: routerA}]

Your content goes here. Edit or remove this text inline or in the module Content settings. You can also style every aspect of this content in the module Design settings and even apply custom CSS to this text in the module Advanced settings.

To demonstrate the effectiveness of Slurp’it, I decided to conduct a practical experiment. My goal was to diagram the network topology automatically, using the tool to collect Link Layer Discovery Protocol (LLDP) data and then process it to visualize the discovered network.

our basic topology

LLDP (Link Layer Discovery Protocol) is a network protocol that helps devices on a network learn about each other. Devices may share their local information and receive information from neighboring devices when enabled. This data exchange allows network administrators or management systems to reconstruct the network’s topology, simplifying network management and troubleshooting tasks. For example, here router PE1 has discovered router PE2 on interface 1/1/c2/1, and the remote port name is 1/1/c2/1.

*A:PE1# show system lldp neighbor     
Link Layer Discovery Protocol (LLDP) System Information

===============================================================================
NB = nearest-bridge   NTPMR = nearest-non-tpmr   NC = nearest-customer
===============================================================================
Lcl Port      Scope Remote Chassis ID  Index  Remote Port     Remote Sys Name
-------------------------------------------------------------------------------
1/1/c2/1      NB    FC:1E:FF:00:00:00  3      1/1/c2/1        PE2
===============================================================================
Number of neighbors : 1

About Slurp’it

Slurp’it utilizes cutting-edge technologies to streamline network data management effortlessly. With its intuitive interface and advanced automation capabilities, Slurp’it simplifies the process of collecting, storing, and processing Command Line Interface (CLI) data. Experience the ease and efficiency of Slurp’it as it transforms the way you manage network data.

The installation is very simple, and after logging in the first screen is a tutorial.

Then we can add our devices in several ways, for now let’s do it manually, we only have two:

Just fill up the necessary fields and check if is SSH reachable and try to SSH login:

testing a new device

We are almost ready to start collecting LLDP information from these Nokia devices.

Let’s check the LLDP planning:

no Nokia SROS native support

So, your favorite vendor is not in the list of supported devices? no problem.

For this first approach, we will adapt to the tool, let’s see how this tools models LLDP.

LLDP report columns

Let’s create a template to catch that information from our CLI output.

Nokia SROS “show system lldp neighbor” basic textFMS template.

now we add this template to the planning:

In Report columns is where we can map our Values defined in the template to the variables that slurp’it is already using. This point is important because it shows that you can preserve the vendor’s own “jargon” during parsing, in modeling this point is very important.

Mapping values

Now let’s schedule the task and

Scheduling collection

LLDP collected data.

Time to automate

With Slurp’it leveraging Swagger’s OpenAI specification, we can automate data collection efficiently and in a structured manner.

a python request.

and then we can use the result of the calculation to diagram the topology.

Slurp’it is communicating with the network, sending SSH commands and receiving CLI output, parsing it and storing it. Topo builder consumes this structured information and is responsible for calculating the topology and delivering a JSON representation of nodes and links. (I chose python and flask for this)

Plotter then consumes the topology and is responsible for exposing an HTML service that the user can access to view the topology (with some cosmetic additions) (JavaScript here)

it’s just a PoC

a simple plotter based on API calls to slurp’ it

Conclusion

The realm of automation tools resembles a dense jungle, where identifying the precise role of each tool can be challenging. Many off-the-shelf solutions boast broad feature sets, yet falter when confronted with complex scenarios. Adapting your existing network to such tools isn’t always feasible, and requesting new features can be frustrating, if not impossible, depending on the tool’s architecture.

Exploring Slurp’it through this PoC has been enlightening. I’ve tested its adaptability, and exposing the gathered data via an API reveals significant potential.

However, this adaptability comes with a drawback: certain workflows may prove more intricate to implement. For instance, repeatedly collecting all LLDP information from scratch during each refresh might not be practical. Instead, focusing on processing only the changes and executing subsequent commands based on that data could be more efficient. This added complexity now requires management at the client side.

Nonetheless, Slurp’it shines in one aspect— It does one thing, and it does it very well.

You can find Part 2 of this POC in the next Blog post by clicking here. In part 2 Flavio will try some more advanced topology mappings.