Photo Storage and Back-Ups

Share

Pro photographer Chase Jarvis has this video on his blog showing his workflow and his approach to data back-ups and redundancy.  I’m not embedding the video here but I you should take a look at this, on his blog or on Youtube (though you’ll miss what he wrote to accompany the video).

The thing is, as someone who has worked in IT for more than two decades, and as someone who started out his career as a system administrator and a database administrator, there’s nothing here I don’t already know.  But I suspect this is news to a lot of people.   There are many bullet points contained in this video, and he talks really fast, but pay attention to how he maintains multiple copies of his original stills and videos separate and apart from the processed/edited versions.   And massive data redundancy/protection, up to and including off-site storage.   This is a lesson that almost everyone learns the hard way (myself included).  But I tell you this, if you do learn it the hard way, one mistake is enough to teach you the proper way for the rest of your life.  My mistake, somewhat fortunately, came early in my career, 22 years ago, and I never made that mistake again.

I don’t do off-site storage now for my personal stuff.  It’s just too expensive.  But most of my data, including all of my photos, is on RAID1 drives.  They’re cheap enough and easy enough to set up that there’s almost no excuse not to do this.  My C: drive isn’t RAID (just slowed things down too much for me) but I do a nightly automated back-up to a RAID drive.

If your data is important enough to you, you do this.  Pure and simple.

It was probably not the best idea for me to watch this video right before heading out to the Wanchai Computer Centre.  I’m running 6 RAID boxes, various capacities, all USB 2.0.  I thought I’d see what my options were for consolidating at least some of this.  One problem is that there was nothing available here that contained multiple drives in one unit that is also USB 3.0.  And at this point, it seems silly to me to spend money on more USB 2.0 devices.

Also it seems that if I want to go for massive amounts of storage (and by massive I mean somewhere around 5 to 10 terabytes) in a RAID configuration, the only drives that will handle that are NAS boxes, and I don’t need that networking component, which adds a lot to the price.

And so, my wallet was spared.

Share

15 thoughts on “Photo Storage and Back-Ups

  1. Shaky

    I built my backup storage using ZFS on OpenSolaris. Cheap mobo, CPU and ram in a case with lots of drive bays and fans. I put 3x1TB and 3x2TB in raidz formation for a total of 6TB usable. Can sustain 1 or sometimes 2 drive failures per pool.

    Bit of a faff to set up when you are learning OSol and zfs, but I rebuilt it all in a few hours once I knew.

    I’ve had one drive fail. You just swap it out and type one command and in a while it’s back to how it was. No data loss.

    I backup to my timemachine drive for my main drive so that happens every hour. For my photo drive, I back up nightly to my zfs box.

    1. Spike Post author

      You’re much more tech than I am these days. All I can remember about Unix is how to spell it. That’s a lot of work just to back-up your collection of Lulu quotes – but then again, they are priceless!

  2. Richard

    Spike I picked up a small 320gb external drive from wan chai (maybe $400 or so?) and sync my photo collection to that once a month. The drive lives at work and I take it back and forth in my pocket – simple offsite backup. Also flickr pro account acts as distributed storage (obv jpegs only so really last ditch) and only US$25 a year.

    1. Spike Post author

      Yeah, I’m on Flickr Pro too, but I don’t upload everything there and I only upload photos after they’ve been edited, not the raw original photos themselves. I have to say, I think there is a big hole in your strategy, unless you’ve only described a part of it. These portable external drives are great for certain things but they are too easily damaged or lost. I would never let one of these drives be the core of my back-up strategy – it would just be one component of a larger chain, although any data redundancy is better than none! My photos exist in several places in the real and the virtual worlds but the original stuff all resides on a RAID1 drive (also relatively cheap these days) – the original RAW file and all of the subsequent data in Lightroom catalogs. And I do realize that I am at risk of losing it all due to fire or theft.

  3. Richard

    My photos are backed up on two separate external drives at home on a daily basis. Not a RAID set up so still imperfect. I bought the external portable as it occurred to me if our block burnt down/was robbed I was pretty much screwed.

    I’ve been toying with a Drobo and S3 for a while but never done anything.

    What I REALLY want is to replace my MBP internal drive with one of the new Seagate SSD/hard drive hybrids but they don’t seem to have hit hk yet (as of three weeks ago anyway). Not a back up strategy so much as big performance improvement for editing raw files at 1/10 of the cost of an SSD.

    1. Spike Post author

      I would sooooo love to go to an SSD for my C: drive, it’s just beyond my budget right now.

  4. Geoff

    I keep two external drives as backup, one kept at home and one at my office. Every month or so, I update the home one and bring it to work, and bring the work one home and update that too. The ‘live’ copy of data is on my iMac.

    You are right about the risk of failure tho…. I had one of the external drives fail and was packing sh*t until I got another backup going.

    1. Spike Post author

      I don’t remember exactly, I’m sure the info is in E-Zone every week but I don’t have any issues lying around the house. I think it’s on a range where 256g ssd costs same as 1t old-style drive. My guys at the computer center tell me the price hasn’t come down at all in a year.

  5. Richard

    Spike check out the Seagate Momentus XT.

    http://www.anandtech.com/show/3734/seagates-momentus-xt-review-finally-a-good-hybrid-hdd

    It’s not SSD speed but in many situations it’s twice as quick as a standard notebook drive, and 500Gb sets you back only USD150. I desperately need a new drive in my notebook (200gb currently) and I’m not prepared to spend the USD1,000 for a pure SSD.

    The folks at the WCCC expected it to hit HK late June/early July, although some people already claim to have it. (http://www.linkecomputer.com/products/1008957/ST95005620AS/Seagate/)

  6. Geoff

    Funny thing… After posting on this topic yesterday, I thought I’d backup my recent work onto the external drive – and the drive failed mount… So there goes one of my backups!

  7. Mark

    You can use the S3Fox extension for Firefox to upload all your photos to Amazon S3 cloud storage. Price is US$0.15C PER GB per month, and its replicated within the cloud for that price. There’s an additional charge for the data transfer, although I believe it may be waived for the first 50TB right now. You can choose US, UK or Singapore pods.

    Everything remains private, with good security, although you can choose to share publicly if you wish. Its not a Flickr alternative – its only storage.

    1. Spike Post author

      Thanks Mark. Yet another reason I admire Amazon and Jeff Bezos, going places you wouldn’t have expected them to go and then doing it so damned well at least most of the time.

  8. Will

    It had actually been a while before I saw Chase’s vid that I decided I needed to invest in some form of redundant storage for my images. Although I run Time Machine on my MacBook Pro, it didn’t stop me losing some work. So, in addition to Time Machine backing up my MBP’s internal drive, I also now have a Drobo with 4TB of protected storage (configured in BeyondRAID – their own system) connected via FireWire 800 as my primary file store.

    To mirror the Drobo, I have a Western Digital 4TB MyBook II which is daisy chained to the Drobo by FireWire and an incremental back-up runs daily on that.

    I’ve also got a plain old external USB drive with my media files for keeping off-site although I keep forgetting to actually take it out of my home office…

    Prior to this, I had the great idea of using Mozy’s unlimited, remote, encrypted, flat-rate backup system until I discovered just how buggy their software was and also just how painfully slow it was to back-up GB’s of data via UK domestic broadband.

    Anyhow, my currently set-up wasn’t cheap, but at least it’s given me some more piece of mind.

    By the way, hi :)

    1. Spike Post author

      Hi, Will. Only had time to take a quick look at your site so far – nice work! And I see we use a lot of the same kit.

Comments are closed.