Screaming Frog v9.0 / Docker / Debian 8 / Ubuntu Remote Desktop

You would like to setup a remote desktop with Screaming Frog to crawl huge websites (+1 million pages) ?

You only need your crawler to run once a month ? (few hours or days)

You have some basic technical skills and want to discover the power of the Cloud ?

An Ubuntu Remote Desktop in a cloud instance in OVH offers a good  price to performance solution. (1,5 € per day / 2$ or half this price if you take a monthly subscription )

What we will do to get an army of crawler:

  1. Open an OVH account or login
  2. Create a new Cloud Project
  3. Create a Server (an instance)
  4. Specify in advance that we want Docker to be installed (it will make everything super simple to setup)
  5. Install a Docker container containing Ubuntu + A remote Desktop based on NoVNC
  6. Connect to Ubuntu with Chrome or any browser in one click 🙂
  7. Install Screaming Frog with 2 commands
  8. Create a Snapshot in Openstack (= OVH)
  9. Create as many server containing Screaming Frog in just ONE clic

Setup a new Cloud Instance

  • Go here and create an account / login: https://www.ovh.ie/public-cloud/instances/
  • Then here: https://www.ovh.com/manager/cloud/index.html
  • Order > Cloud Project > Fill the forms
  • In your project > Infrastructure > You can add a new server in “Actions” > Add Server
  • Take the 7GB RAM & 100GB SSD for this test.
  • You will need 60 GB of disk for 1 Million URL Crawled
  • Setup your ssh key (Google is your best friend to get help, it’s OS specific)

Setup the server

  • Connect to the server with a terminal
    • user@IP of your server
  • Then copy and paste each line one by one:
    • apt-get update
    • apt-get upgrade
    • sudo docker run -it –rm -p 6080:80 -p 5900:5900 -e VNC_PASSWORD=MyPassWordToReplaceByWhatYouWant dorowu/ubuntu-desktop-lxde-vnc

Setup Screaming Frog

  • Connect to https://IP-OF-YOUR-SERVER:6080/ with the password you used for “VNC_PASSWORD=”
  • Open a terminal in Ubuntu (in the NoVNC session – icon in the bottom left of the Ubuntu desktop)
  • Then copy and paste each line one by one:
    • sudo apt-get install screen wget
    • wget https://download.screamingfrog.co.uk/products/seo-spider/screamingfrogseospider_9.0_all.deb
    • dpkg -i screamingfrogseospider_9.0_all.deb
    • sudo apt-get -f install
  • Screaming Frog is now installed 🙂
  • You can try it here:
    • Bottom left icon > Internet > Screaming Frog SEO Spider
  • You will have to setup the Storage to disk to crawl huge websites
    • Screaming Frog > Configuration > System > Storage

Next step: create a snapshot of this in OVH (to be continued if you liked this article !)

Links to find Bicycling Roads in Croatia

Here are 3 links which will help you to discover Croatia with your bicycle:)

A super personal website:

http://www.takeadventure.com/

Google Earth KML track:

http://www.parenzana.net/en/multimedia/map-gps/5673 

You might need a Croatian friend of Google Translate for this one:

http://www.pedala.hr/category/biciklisticke-rute-staze/biciklisticke-rute-staze-zupanije/

La liste des sites équivalents Le Bon Coin en Europe et ailleurs

Voici la liste des sites équivalents au site de petites annonces Le Bon Coin (également appelé Leboncoin ou leboncoin.fr).

La liste a été mise à jour le 25 juillet 2020.

Petites annonces Allemagne

  1. https://www.ebay-kleinanzeigen.de
    1. annonces voiture sur ebay
      1. Volkswagen  (131 686)
      2. Mercedes Benz  (72.191)
      3. BMW  (70.029)
      4. Audi  (56 469)
      5. Renault  (27.131)
      6. (…)

Petites annonces Angleterre

  1. Londres: https://london.craigslist.co.uk/
  2. https://www.gumtree.com/

Petites annonces Autriche

  1. https://www.ebay-kleinanzeigen.de/s-buecher-zeitschriften/c76

Petites annonces Croatie

Continue reading “La liste des sites équivalents Le Bon Coin en Europe et ailleurs”

Linux shell bash colors : the easy way to change them

#Backup your dircolors file : dircolors -p > ~/.dircolorsrc #Change the color .dircolorsrc : nano .dircolorsrc #If you want to change the directory color to yellow, find DIR 01,34 and change it : DIR 01;33 # directory #Change the .bashrc values : eval “`dircolors -b ~/.dircolorsrc`” export LS_OPTIONS=’–color=auto’

Enjoy!

Social Media Manager : How To Be REALLY Successful In SEO

My girlfriend is Social Media Manager and Content Marketing Manager as Freelance for several international brands in Croatia. She asked me today some advices to improve the SEO of one of her clients (I’m SEO Expert since 2009).

If you had to remember one rule as Social Media Manager, when embedded in missions with potential SEO impact, keep in mind one rule, the best trick ever:

Think for the user, start from when he wonders something, then when he makes a research in Google, to the moment you would like him to leave your website.
That’s best SEO practice ever. That’s how your SEO will work. The more your content will be appreciated, the more it may improve your SEO. Enrolled in a linkbuilding mission? Accept only links from high quality content and website!
That means, as Social Media Manager and Content Marketing Manager, spend almost all your time to IMPROVE CONTENT QUALITY and User Satisfaction / User Experience. And you know what? You’ve been hired for that, and you are very good at doing that, I’m sure 🙂

If this rule is too hard to follow, then do at least some easy optimisations:

  • Add the mains keywords naturally in the beginning of you Title, and put it also in the meta description, H1 and in the  main content at least one time in each of them is a good rule. One time is enough in most cases ! Put some variation of the targeted keyword in the body and in the H2, if you want.
  • <Title> : you get found in Google because the users recognize that this page talk about what they are looking for. The same for Meta Description.
  • <H1> : Put it in it, so user knows that he is arrived in the good page, which talks about what he is looking for.
  • <Body> : So Google knows for sure that your page talks about the main keyword you try to rank for.
Mariana asked my opinion about this article.

Here is one part of this article :

SEO magic happens when you identify one keyword to focus on, then put it in the trifecta of permalink, page title, and page description.

Please, don’t write this in 2015 and even less in 2016 (edit: and even less in 2018 !).

1/ SEO rules have changed, and there is, no more than before, any “magic” in SEO. Until now, SEO is based on algorithms, but no magic, please.

2/ URL trick doesn’t work, except for pictures, and in this case, it’s like 0,001 % important.

So, again :

Think for the user first, start from when he wonders something, then when he makes a research in Google, up to the moment you would like him to leave your website.
Looking for some reliable informations about SEO? Start following Rand Fishkin advices, what he says is easy to understand. This guy deserves his popularity in the SEO community.

How-to install mnoGoSearch on Debian Jessie

mnoGoSearch tested with a fresh cloud.runabove.com Debian Jessie 7.5 instance :

sudo nano /etc/apt/sources.list
#Only jessie distrib
---> deb http://ftp.debian.org/debian jessie main
---> deb-src http://ftp.debian.org/debian jessie main

#Update package list
sudo apt-get update

#Upgrade distrib with new distrib repository
sudo apt-get upgrade

#Install MySQL & PhpMyAdmin BEFORE mnoGoSearch
sudo apt-get install mysql-server phpmyadmin

#Create a new db
---> create new db db_test_mnogo at http://IP/phpmyadmin/

#Go in the /tmp/ directory for instance
cd /tmp/

#Download mnogoSearch package / Here it's not the last one
wget http://www.mnogosearch.org/Download/deb/mnogosearch_3.3.13-1.static_amd64.deb

#Unpack and install
sudo dpkg -i mnogosearch_3.3.13-1.static_amd64.deb

#Go to the newly created mnoGoSearch directory
cd /etc/mnogosearch

#Backup and rename the conf file
sudo cp indexer.conf-dist indexer.conf

#Setup mnoGoSearch to work with MySQL
sudo nano indexer.conf
---> replace DBAddr  mysql://root:passmysql@localhost/db_test_mnogo/?dbmode=blob
---> add Server http://www.website-i-want-to-crawl.com/ near the end of the file

#Go to the exe file directory
cd /usr/sbin/mnogosearch

#Create DB
./indexer -Ecreate
#run
./indexer
Enjoy :)

Official doc : http://www.mnogosearch.org/doc33/msearch-indexing.html

How-to Install Elasticsearch on Debian 7 OVH VPS

install python-software-properties :

apt-get install python-software-properties

install a non official repository Oracle Java :

echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee /etc/apt/sources.list.d/webupd8team-java.list
echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu precise main" | tee -a /etc/apt/sources.list.d/webupd8team-java.list
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys EEA14886
apt-get update

install Oracle JDK 7

apt-get install oracle-java7-installer

Download and install the Elasticsearch Public Signing Key

wget -qO - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -

Add the following to your /etc/apt/sources.list to enable the repository

deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main

Run apt-get update and the repository is ready for use. You can install it with :

apt-get update && apt-get install elasticsearch