Friday, March 2, 2012

How to backup my data (and why!)



It is the digital age, the amount of personal data we produce keeps going up: digital pictures, HD movies and documents take an ever increasing amount of space.
That's a lot a memories and information we don't want to loose (or can't afford to).

Companies have devised backup plans for a long time but the concept is now entering homes though cloud storage (and other means). When your data is lost, it is too late: you need to devise a plan now!

I will focus on the needs of individuals and deal with 3 different data types:
- photos
- videos
- documents (excel, doc, text, pdf...)

Also, there are diffrent risks to take into account when defining a backup plan:
- hardware failure (crashed hard drive)
- physical destruction of data at a physical location (think fire, theft ...)
- human error (oops! I deleted the file)


1. Put your data in the Cloud

The cloud will shield you from hardware failure, physical destruction but might not protect you against human error.... Added bonus is that you can share your data with other people :)
You also take on additional risks: like the risk of you online account being hacked or the risk of your data becoming visible to everybody because of a misconfiguration on your side.

The good news is that there usually are free allocations for each service but you might have to pay for a feature you really need.

For pictures, you have:
- Picasa: 1GB free + free unlimited storage of pictures up to 800x800 pixels (additional storage available: cost of 20GB is 5USD/year, see all prices here)
- Google plus: free unlimited storage of pictures up to 2048x2048 pixels
- Flickr: free upload of 300MB worth of pictures every month, paid option unlimited storage (original quality) & bandwith for (25 USD/year or 45 SD/2y. see all prices here)


For movies, you have:
- Youtube: Videos can be uploaded for free (up to 20GB per video)
The problem is that the videos are automatically edited (and reduced in quality) and it is not easy to download them once they are in the cloud!


For documents, you have:
- Google Docs: storage of documents, presentation and spreadsheets in google format is unlimited, you get 1GB for other type of files. Additional space can be bought (and shared with picasa). See the above link in Picasa for pricing details. The problem is there is no easy way to synchronize a local folder with Google Documents...
- Dropbox: 2B free storage. Local folders can be synchronized with dropbox.

2. Use a backup Service

This will shield you from hardware failure (but it might be slow to recover the data), physical destruction and human error.

The idea is to send your compressed and encrypted data to a remote server where it is stored. You can usually access our backups from a website and from a specific software.
The problem is that all your data goes through the internet and it can be very slow to do the initial upload if you have a lot of data. For example, if you have 1TB of data to backup, it can take months to do the initial backup!
Same problem when you try to do a full backup: it will be usually an order of magnitude faster than doing the initial backup but it can still take a few days.


If you also need to recover fast from an hard drive failure (the most common hardware failure) you can use a local redundant RAID configuration (like RAID 1 or 5). Please note that RAID alone will not prevent data loss: you are still vulnerable to other hardware failures (like RAID controller failure, destruction of the device and human error).

Let's compare the different plans out there. I will focus on 3 providers: Mozy, Carbonite and Crashplan.
Plan Name +10GBUnlimitedFamily unlimited HomeHomePlusHomePremier 50GB125GB
Yearly Price 25 USD50 USD120 USD 59 USD99 USD149 USD 72 USD120 USD
Space 10 GBUnlimitedUnlimited Unlimited 50 GB125 GB
Number of computers 112-10 111 1 (add computer +2USD/month)
Suported Os Windows, Mac, Linux, Solaris Windows, MacWindows Windows, Mac
Automated backup All files All Except videoAll files All files

Whatever your usage, Crashplan seems always cheeper and has more features. I use it myself and I am very satisfied with it....


The free crasplan software also allows you to backup on a friend computer (running crashplan as well). This means that you can backup your data without paying anything, provided a friend is ready to allocate you some data for backup.


3. Case studies

We just need to find the most cost effective combination of the above:

Profile A:10GB of pictures and a few documents: 5USD/year
Pay 5USD/year for Picasa storage (20GB)
Use DropBox free allocation to store the documents.
Problem: the backup process is manual: if you forget to upload your pictures to Picasa, they are not uploaded, unless you use the software I wrote to automatically upload to Picasa: see my post here). You are still vulnerable to Human error.

Profile B:100GB of pictures, 200GB of movies and a 10GB of documents: 50USD/year
Cheapest alternative is Crashplan Unlimited (50USD / year)
The backup process is now automatic: no need to worry about forgetting to backup something. On top of that, you are protected against human error as you can retrieve former versions of a file.

If your data is spread across different computer, you can buy a NAS and run crasplan on the NAS (see my post on how to install crasplan on an Iomega NAS here. Alternatively, you have the simpler option to buy Crashplan unlimited Family.


4. Conclusion

If you care about your data: take the time to devise your backup/data recovery plan now! You can always find a way that fits your budget.

You can get a reduced quality backup for your pictures and videos for free. Truct me, it is better to have a reduced quality backup than nothing!


Thursday, March 1, 2012

How to install Vuze on a NAS




The goal of this tutorial is to install vuze headless (as a command line application). Most of the tutorials found on the web suggest to do the configuration of vuze in the UI before starting it in headless mode. Unfortunately, this is not possible on a NAS where you have to X server and no screen...



Tutorial tested on IOMEGA Storcenter ix4-200d firmware 3.1.14.995 but uses unsupported features on the hardware. Please use at your own risk.


Since Vuze if a java program, the same steps should allow you to install vuze as an headless client on any hardware running java.




Unfortunately, I ran into a lot of JVM crashes with vuze headless and oracle jvm ejre1.7.0 (for ARM). On top of that, vuze is quite an heavy program in terms of CPU and memory usage, which is annoying for the type of hardware we are looking at (like a NAS). Therefore, I don't recommand to install vuze on a NAS. I suggest you look at rtorrent, which is much more reliable (see my tutorial How to install rtorrent with IP filtering).





1. SSH into your NAS

See my other post:
How to ssh into your Iomega StorCenter ix4-200d if you have an IOMEGA NAS




2. Download and install

Steps adapted from the Console_UI Vuze wiki
cd /opt/tmp
wget http://ftp.heanet.ie/mirrors/www.apache.org/dist//commons/cli/binaries/commons-cli-1.2-bin.tar.gz
tar 
wget http://sourceforge.net/projects/azureus/files/vuze/Vuze_4702/Vuze_4702_linux.tar.bz2
PATH=$PATH:/opt/bin/
tar -xvf Vuze_4702_linux.tar.bz2

wget http://ftp.heanet.ie/mirrors/www.apache.org/dist//commons/cli/binaries/commons-cli-1.2-bin.tar.gz
tar -xvf commons-cli-1.2-bin.tar.gz
mv commons-cli-1.2/commons-cli-1.2.jar vuze/

wget http://ftp.heanet.ie/mirrors/www.apache.org/dist//logging/log4j/1.2.16/apache-log4j-1.2.16.tar.gz
tar -xvf apache-log4j-1.2.16.tar.gz
mv apache-log4j-1.2.16/log4j-1.2.16.jar vuze

to install java, you can look at the java section of my other post: How to install crashplan on an Iomaga NAS.

The installation is pretty straight forward...



Now, install the webUI plugin:
cd vuze
cd plugins
mkdir webui
cd webui
wget http://azureus.sourceforge.net/plugins/webui_1.7.0.zip
ipkg-opt install zip
ipkg-opt install unzip
unzip webui_1.7.0.zip
mkdir /opt/var/log/vuze
If you don't have ipkg-opt, see my other post: How to install software into your Iomega StorCenter NAS



3. Configure the Vuze installation

cd /opt/tmp/
mv vuze /opt/
cd /opt/vuze/
/mnt/pools/A/A0/NAS_Extension/ejre1.7.0/bin/java -Xmx128m -Dazureus.config.path=/opt/vuze/.azureus/ -cp "Azureus2.jar:commons-cli-1.2.jar:log4j-1.2.16.jar" org.gudy.azureus2.ui.common.Main --ui=console
Vuze should now be running, we need to configure it now. Adapt the paths to suit your needs and type at the vuze cli:
set "Default save path" "/mnt/pools/A/A0/torrents/vuze/download" string
set "Use default data dir" true boolean
set "Logger.Enabled" true boolean
set "Logging Enable" true boolean
set "Logging Dir" "/opt/var/log/vuze/" string
set "Ip Filter Autoload File" "http://list.iblocklist.com/?list=bt_level1" string
set Plugin.azhtmlwebui.User myusername
set Plugin.azhtmlwebui.Password mypassword password
set "Plugin.azhtmlwebui.Password Enable" true boolean
This is installing IP filtering as well. If you don't want that, just skip the set "Ip Filter Autoload File" command.

You are good to go now. To have vuze automatically start at boot, you need to create the script in /etc/init.d (you can adapt the azureus script provided inside the install).
If you have an Iomega NAS, look at this tutorial to see how to have the program run at boot.





You can now connect to the web Vuze UI from your web browser at
http://ip_of_nas:6883/. Please note that the web UI is not as rich as the regular UI (most options are not available in the web UI)




Please comment to let me know how stable your install is!

Thanks