Upgrade MVP to 1.0.x

This major update requires a bit of changes for your running setup.

Architecture changes
We created a new service (docker image) which will handle all the database requests. This service is called Warehouse. For this reason it’s required to pull the latest changes from our Git repository: https://gitlab.com/slurpit.io/images
This will probably create merge conflicts since you changed the existing docker-compose.yml file.

For future changes we decided to include a overwrite file so it’s easier to upgrade your existing project.

So what do you need to do?
We support 2 options here, if you don’t care what happens to your existing collected data or if you want to migrate your existing collected data.

Backup
In case you want to make a backup before you start to migrate the Databases, keep in mind that it’s important to preserve the original file permissions. To do this you have to include the -a in your copy command. For example:

cp -a /opt/slurpit /opt/slurpit_backup -R
-a means keep the original permissions
-R means copy every file in the folder

1. I don’t care what happens to my existing collected data

• Turn off the system by running sh down.sh
• Rename the existing project folder, for example:
  mv /opt/slurpit /opt/slurpit_backup
• Copy the updated project from git:
  git clone https://gitlab.com/slurpit.io/images.git /opt/slurpit
• Rename the override file
  mv docker-compose.override-EXAMPLE.yml docker-compose.override.yml
• Make the changes you made before in the docker-compose file in the docker-compose.override file
• Copy the portal database from the old environment to the new slurpit environment.
  Reason we keep the portal database is that because here is all our logic stored so this is not something we want to lose.

  cp -a /opt/slurpit_backup/db/portal /opt/slurpit/db/portal

  Note:
  – It’s important to include the -a to make sure that the original permissions are preserved
  – If the db folder does not exist yet you can create it with mkdir db

• Start the system again by executing: sh up.sh

2. I would like to keep my existing Collected Data

Depending on the server hardware we can migrate the collected data from the old environment to the new one.
We have seens servers with a database of more then 50GB with only 8GB RAM which were not able to migrate the database.
In this case it’s better to use the option above where we don’t care about the Collected Data.

• Turn off the system by running sh down.sh
• Rename the existing project folder, for example:
  mv /opt/slurpit /opt/slurpit_backup
• Copy the updated project from git:
  git clone https://gitlab.com/slurpit.io/images.git /opt/slurpit
• Rename the override file
  mv docker-compose.override-EXAMPLE.yml docker-compose.override.yml
• Make the changes you made before in the docker-compose file in the docker-compose.override file
• Copy the db folder from the old environment to the new slurpit environment.
  

  cp -a /opt/slurpit_backup/db /opt/slurpit/db -R

  Note:

  – It’s important to include the -a to make sure that the original permissions are preserved
  – If the db folder does not exist yet you can create it with mkdir db

• Remove the Scanner database
  rm -rf /opt/slurpit/db/scanner
• Rename the Scraper database
  mv /opt/slurpit/db/scraper /opt/slurpit/db/warehouse
• Start the system again by executing: sh up.sh
  Keep in mind that the migration can take a while depending on the database size