Architecture

The Slurp’it architecture is build and based on years of experience and customer feedback. When I worked in the Airplane communication field the rule of thumb was, technology should have proven itself for at least 7 years before we used it. We applied the same on our team, it consists of developers who worked for at least 7 years on Big Data software or Network Automation software. This combination is a gem for Slurp’it.

To give you a bit of a background, late 2023 we launched the MVP release of Slurp’it. The NetDev community jumped on top if it and gave us an amazing production network to test Slurp’it on which was bigger then we could ever imagine.

Working with networking devices always brings some kind of challenge, there is not really a virtual test networking framework for virtual Network Devices except Eve-NG. But to fully manage 117 vendors and rebuilding them when something breaks is though with Eve-NG, therefor we build our own Open-Source Device Imitator (which you can download and contribute to here). So to stay on topic, we use our own device imitator in the CI/CD pipelines to make sure Slurp’it works on all delivered content.

Container architecture

The Slurp’it underlaying architecture is Micro Service based on a Container Network. Our prefered way is Docker, but it’s also possible to run Podman or Kubernetes. Why did we go for a Container Architecture? well for one, it’s really flexible. You can easily change port numbers, enlarge poolsizes for multithreading but also have the services run on different servers. And best of all, the volumes are just a data folder. So to make a backup you only have to zip or tar the volume (make sure you preserve the user rights).

No experience with container based applications? don’t worry we setup a OVA for you on this page.

Slurp’it architecture

The basic setup contains out of the following containers:

• Portal
  The GUI, Rest API, SQL Database, Scheduler & workers.
• Warehouse
  Stores all the raw data from the scanners and scrapers in a NoSQL database (MongoDB).
• Scanner
  Is the container for the Device Finder & Crawler. By using SNMP and a L2 & l3 scanner it will try to find the devices on your network and recognize them by using SNMP
• Scraper
  Is the container for the service Data collector which will use the templates and planning information from the Portal, to grab raw device data and store it structured in the Warehouse.

 All the communication through the containers happens by individual REST apis, Therefor you can easily have the containers run on different servers. On the same server it will use the Internal Network of the container application.

Simple overview

Architecture flow

Horizontal scaling

But what if you have a very large network with security zones where it’s not possible for the services, e.g. Data Collector & Device Finder to connect to different types of networks?

For this reason we made it possible for our Enterprise customers to have multiple services running in different networking segments.