For the record, I know about keeping good backups, but sometimes stuff just happens in the perfect sh#t storm and you lose your data. That’s what happened to me a couple of months ago. Since then, I learned a lot about file recovery and the enormous investment of time and effort it takes to get back just some of the files. I also learned a little about letting go. Most of those lost gigabytes were files that I was meaning to get around to, but I just never had the time to sort through the virtual pile. That’s one way to clean house, but I wouldn’t recommend it.
In my case, the data was lost due to a formatted drive, and so the files had to be recovered using file signatures. I was able to retrieve thousands of files, but most of them didn’t have proper file names, so there was no folder structure to give a file its context. That’s okay for pictures but not great for source code. The worst part was not knowing what was lost, although I now feel pretty good about what I was able to get back. When all of my data went away, I quickly thought about the important stuff that was on my drive. There are a few documents, but really it all came down to the irreplaceable pictures and videos. That is what keeps you up at night, running through scenarios of how all of this could have been prevented, but what’s done is done and it does no good to dwell on things that can’t be changed. Data recovery was made more complex because my drive contained backups of my wife’s computer, so I had to deal with hundreds of thousands of duplicate files.
Throughout the file recovery process, I found a few fantastic tools which did a lot of the heavy lifting for me. These tools are awesome on their own and as a bonus, they are free! I could not have retrieved my files and sorted out the mess without them. So, given that I did a lot of research and testing, I thought I would present you with the best of breed software for data recovery and backups. If something like this ever happens to you, these programs can help you get back your data without having to spend hundreds of dollars and countless hours finding the right tool.
Please Contribute to the Authors
The software I recommend below is not a watered-down version of a commercial product. This software is awesome and absolutely free, but if you find it useful, please donate to the authors. It takes a lot of time and effort to write high quality programs and these talented folks deserve to be compensated for their hard work.
Best Data Recovery Tool
Getting your data back all starts with good data recovery software. When I lost my data, I initially sought commercial software believing that it would be higher quality than the free stuff and would provide me with a more complete recovery. That erroneous belief was dispelled after a few failed attempts at using the commercial product. These programs tend to be expensive, starting at $100 or more. Data recovery is time consuming too, so you want to choose a product that doesn’t waste your time by returning incomplete or corrupted data. For my particular data loss situation, I needed something that could recover files using file signatures. I discovered a truly remarkable and free piece of software called PhotoRec by CGSecurity. The software has a bit of learning curve, but don’t be scared off by the console interface. This utility scourers your hard drive for a wide range of file types that it can detect (for a complete list of file formats, go here).
For a detailed step-by-step guide to using PhotoRec, go here
I should warn you that this kind of file recovery takes hours and sometimes days to complete. When the process is done you’ll have a whole mess of directories (dir.001, dir.002, dir.912, etc) to sort through, and that is when you’ll need a good file duplication finder which brings me to my next best in class category…
Best Duplicate File Finder
In my experience, the absolute best file duplication finder software is:
dupeGuru Picture Edition
When recovering data from large backup drives, you will really need a great file duplicate finder; one that is fast but accurate. DupeGuru is exceptional in that it is able to find all of your dups quickly and accurately. It has intelligent file selection, allowing you to weed out the dups without accidentally deleting both files.
Figure showing a typical duplicate file scan
Once the scan is complete, you will be presented with a results window showing a list of duplicates. From there you can easily delete the duplicates by choosing Edit–>Mark All and then Actions–>Send Marked to Recycle bin. However, it is a good idea to review the results to ensure that you aren’t deleting files that are in fact different photos that only highly resemble another picture. In my experience this only happens in very dark photos or when the pictures were taken in rapid succession and not much changed between shots.
Best Cloud Based Backup
(Tentative recommendation-see update)
Okay, the cloud based (centralized) version of CrashPlan is not free, but the good folks at Code42 do offer a free Computer-to-Computer backup solution that works great if you have a friend or relative who are willing to host your backup drive for you and in kind, you could host a backup drive for them. All of the data is encrypted, so no one can read files on the offsite hard drive. Although this option is really great, I chose to go with the centralized CrashPlan solution.
For me, CrashPlan is the best online backup solution.
When performing a restore, I encountered several “Integrity check failed” errors. It seems that the CrashPlan agent occasionally encounters errors backing up files but does not report the error or perhaps is even aware that some files are corrupt. I found a blog post where someone else had a similar experience (see: http://try-dot-ch.blogspot.ca/2010/03/crashplan.html) There is no easy way to tell CrashPlan to re-transfer corrupted files but worse still, the agent doesn’t notify you of files in your backup set that are corrupt. I thought that overtime the files would “heal” but the integrity errors have persisted for months now. Obviously, I cannot wholeheartedly endorse a product with such a flaw. I have opened a ticket with Code42 to see if they can remedy the matter. I’ll keep you posted.
After working with the capable support staff at Code42, they were able to fix my file recovery issues. My support experience was excellent and I can now wholeheartedly recommend CrashPlan as a terrific cloud-based backup solution.
I tried other products like BackBlaze, but I recommend CrashPlan for the following reasons:
- Unlimited backup of a single computer, including external drives.
- Super-smart, yet unobtrusive backup agent that intelligently scans your system for file changes.
- Economical cloud-based solution
- Easy to use interface that makes backing up and restoring a snap. I restored a very large photo database without issue.
- Keeps all versions of a file, not just the last 30 days. This is great if an important document gets deleted and you didn’t notice until six weeks have past.
- You almost wouldn’t know that the app is there keeping all of your data backed up.
- CrashPlan is secure, providing 448-bit encryption on disk and 128-bit encryption during file transfer.
- Works on PC, Mac and Linux
The cloud is your friend. When you data lives in the cloud, you are far less likely to accidentally delete your entire collection of files.
So there it is. In the very least please consider using a cloud-based backup for your files.
Oh, I almost forgot another key part of my new backup solution. Having my data in the cloud is good, but I also wanted to have a good local backup that would allow me to fully recover my user files AND operating system in case of a hard drive failure. This brings me to my Bonus Software Recommendations:
Best Backup Software
The program I recommend in this category is not free, but sometimes if you want the best, you will have to pay for it. I don’t mind paying a reasonable price for software that is really good. I tried other products like EaseUs backup, but in my experience nothing beats:
Acronis True Image 2015
Here is what I like about True Image 2015:
- The app has an amazing interface that makes creating and restoring backups a breeze.
- Has an intelligent restoration process that actually works. I tried other products that left my Windows 8 computer unbootable after a restore.
- You can schedule full and incremental backups.
- Runs on PC and Mac!
- Backups up data to a network drive
- Comes with a lot of neat tools like Clone disk and Parallels Access.
But don’t take my word for it, you can download the free trial here.
There you have it. Something good can come from a data disaster after all. If you should ever find yourself in a similar situation, I hope my software recommendations and advice help get your data back in good time.