The Slurp’it architecture is built and based on years of experience and customer feedback. When I worked in the Airplane communication field the rule of thumb was, technology should have proven itself for at least 7 years before we used it. We applied the same on our team, it consists of developers who worked for at least 7 years on Big Data software or Network Automation software. This combination is a gem for Slurp’it.

To give you a bit of background, in late 2023 we launched the MVP release of Slurp’it. The NetDev community jumped on top if it and gave us an amazing production network to test Slurp’it on which was bigger then we could ever imagine.

Working with networking devices always brings some kind of challenge, there is not really a virtual test networking framework for virtual Network Devices except Eve-NG. But to fully manage 117 vendors and rebuilding them when something breaks is tough with Eve-NG, therefore we built our own Open-Source Device Imitator (which you can download and contribute to here). So to stay on topic, we use our own device imitator in the CI/CD pipelines to make sure Slurp’it works on all delivered content.

Container architecture

The Slurp’it underlying architecture is Micro-Service based on a Container Network. Our prefered way is Docker, but it’s also possible to run Podman or Kubernetes. Why did we go for a Container Architecture? Well for one, it’s really flexible. You can easily change port numbers, enlarge poolsizes for multithreading but also have the services run on different servers. And best of all, the volumes are just a data folder. So to make a backup you only have to zip or tar the volume (make sure you preserve the user rights).

No experience with container based applications? Don’t worry we setup a OVA for you on this page.

Multi server setup

Benefit of a container architecture is that you can easily scale the application between servers.
The following page explains the benefits of this and has the links toward a setup video and the techinical documentation.
Visit here the Multi Server Setup configuration page.

Slurp’it architecture

The basic setup consists of the following containers:

• Portal
  The GUI, Rest API, SQL Database, Scheduler & workers.
• Warehouse
  Stores all the raw data from the scanners and scrapers in a NoSQL database (MongoDB).
• Scanner
  Is the container for the Device Finder & Crawler. By using SNMP and a L2 & L3 scanner it will try to find the devices on your network and recognise them by using SNMP.
• Scraper
  Is the container for the service Data Collector which will use the templates and planning information from the Portal, to grab raw device data and store it as structured data in the Warehouse.

 All the communication through the containers happens by individual REST APIs, therefore you can easily have the containers run on different servers. On the same server it will use the Internal Network of the container application. Between the containers is a mechanism available called profiling where it’s building a device profile to record the best settings to optimise the communication and connectivity. 

Simple overview

Architecture flow

Horizontal scaling

But what if you have a very large network with security zones where it’s not possible for the services, e.g. Data Collector & Device Finder to connect to different types of networks?

For this reason we made it possible for our Enterprise customers to have multiple services running in different networking segments.

Multi Tenant setup

In theory you could use the horizontal scaling solution to have every tenant connect to the same warehouse. But this could lead to a lot of overhead and a large growing database. Therefore our approach with be to setup the Service Manager where you will give every tenant his own Slurp’it instance which you can manage from a central server.

Benefits

– Easy to add or remove a Tenant
– Customer data is isolated within a Tenant setup
– Backup / Restore or System Maintenance can be planned per Tenant.