How To Screw Up a Good Backup System

I back up our family computers like crazy, using a combination of Time Machine and cloud backup via Backblaze. But I did something dumb and almost lost our family’s entire history of home videos. Facing estimates of $500 – $1500 for professional data recovery, I stumbled on an awesome hack that saved the day.

Background

All of our computers’ internal hard drives get a dedicated external Time Machine, and we use Backblaze for extra insurance, so our data is safe in the cloud in case of fire, theft or flood. But we also have a few external hard drives that store things like large music collections and our home videos. The external drives back up to Backblaze only (no Time Machine).

All of that has been working hunky dory for years, and I felt confident we were safe. Then, a week ago, I realized that the drive that stores our family videos (“Gorgonzola”) would no longer mount, with any cable, on any of our machines. Yikes! So I turned to Backblaze for a restore, only to find it wasn’t showing up there either! Double yikes, freakout.

What Happened

At some point in the distant past, I did something dumb, though I didn’t realize it was dumb at the time – I realized I had lots of extra space on Gorgonzola and decided to let that drive do double-duty, as a Time Machine drive for a laptop. What I didn’t take into account was the fact that Backblaze has a reasonable rule – they don’t back up your backups. So when Backblaze detected that Gorgonzola was now a Time Machine drive, it dropped it from the manifest. I never noticed it had been dropped.

So now I’d realized that I had NO backup available anywhere for this precious, unreplaceable data. So I called some data recovery services, and got estimates ranging from $500 – $1500. The data was important enough to me that I’d pay that ransom, if it came to it. But of course I didn’t want to.

So Crazy It Just Might Work

The next day, I stumbled on a brilliant suggestion: Often, when a drive won’t mount, it’s because the USB controller circuit board inside the drive case has gone south, and that the drive itself is fine. Solution: Purchase an identical drive, take them both apart, and swap the controller chips. Brilliant! Found an identical drive on Amazon for $100 (Seagate Backup Plus Slim), and went for it.

Prying the case open and removing the drives turned out to be easier than expected (YouTube video), and the controller chips slipped off easily. Quick swaperoo, and lo and behold, it worked! Gorgonzola showed up as normal, and I’m rescuing my data right now. Of course, both drives were destroyed in the process, but at this point, I don’t care.

Yay, internet.

Django template tag to display latest git commit, date and tag in template

I found a few references to bits and pieces of this in various places, but this snippet ties all three bits of information together in a single string, and accounts for calling the path to the git repo from anywhere (which tends to work “magically” on localhost but breaks on servers).

If you don’t use git tags, season to taste.

@register.simple_tag
def git_ver():
    '''
    Retrieve and return the latest git commit hash ID and tag as a dict.
    '''

    git_dir = os.path.dirname(settings.BASE_DIR)

    try:
        # Date and hash ID
        head = subprocess.Popen(
            "git -C {dir} log -1 --pretty=format:\"%h on %cd\" --date=short".format(dir=git_dir),
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        version = head.stdout.readline().strip().decode('utf-8')

        # Latest tag
        head = subprocess.Popen(
            "git -C {dir} describe --tags $(git -C {dir} rev-list --tags --max-count=1)".format(dir=git_dir),
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        latest_tag = head.stdout.readline().strip().decode('utf-8')

        git_string = "{v}, {t}".format(v=version, t=latest_tag)
    except:
        git_string = u'unknown'

    return git_string

Then, in your template, you can simply:

{% if user.is_superuser %}{% git_ver %}{% endif %}

GoPro Time Lapse: Stinson Beach

Last weekend at Stinson Beach I attached a GoPro to a tree and had it shoot one image every 5 seconds for a couple of hours. Later compiled the image into a 29fps half-speed video in GoPro Studio. Not sure why I enjoy making these so much; something oddly satisfying about the process.

View 403, 404, 500 with media in Django DEBUG mode

When working with Django in DEBUG mode, it can be tough to see your 403, 404, and 500 views, since they raise visible stack traces instead of the UX the end user will see. But if you turn DEBUG off, runserver’s local media serving is disabled because it’s designed to work only with DEBUG = True. The solution is scattered throughout the Django docs, and I couldn’t find it compiled into one compact code block anywhere – just reference the handling functions directly from the end of your urls.py:

if settings.DEBUG:
    from django.views.defaults import server_error, page_not_found, permission_denied
    urlpatterns += [
        url(r'^500/$', server_error),
        url(r'^403/$', permission_denied, kwargs={'exception': Exception("Permission Denied")}),
        url(r'^404/$', page_not_found, kwargs={'exception': Exception("Page not Found")}),
    ]

And voila, your 403.html, 404.html, and 500.html templates will be displayed in full glory for developers.

Remove Duplicates, Fix Broken Album Art in iCloud Music Library

Apple Music + iCloud Music Library is a brilliant pairing, and finally lets us access our personal music collections from anywhere. But it’s not without its warts – duplicated tracks and bad/missing cover art has been a sore spot for iCloud Music Library users since the service launched. In my first piece for Medium.com, I walk readers through the reasons – and the fixes – for those two problems.

record-collection

The Ethics of Modern Web Ad-Blocking

For years, I’ve resisted – and argued against – using web ad blockers of any kind. After spending a decade working at a journalism school and watching publishers large and small struggle (and mostly fail) to find a way to be paid for their essential efforts, it felt like bearing a certain amount of advertising was the very least we-the-public could do to support quality journalism. Paywalls don’t work for almost any publications – what else is there?

But the rise of the mobile web tipped the scales – the “social contract” around advertising was no longer a fair one. The mobile experience is far less tolerant of intrusion, and network speeds are slower. But because monetization is more difficult, publishers were “forced” to insert more, and more intrusive advertising. The cumulative effect has been a steady decline in the quality of mobile browsing. Today, many news sites are close to unusable on a smart phone, having become choked out by network and screen-stealing crap.
Continue reading

Sane Password Strength Validation for Django with zxcvbn

While many admins and blog posts tell users that length is by far the most important factor in creating strong passwords/passphrases, the majority of password input fields are giving them a set of hide-bound rules: Eight characters, at least one upper- and one lowercase letter, some digits and punctuation marks, etc.

Even though it includes dictionary words, a passphrase like:

Sgt. Pepper's Mr. Kite

is far stronger than:

js72(.Tb8

(there’s a world of difference between 22 characters and 9, from a cracking perspective). But many password input fields would reject the first one. No wonder users are confused by the process of creating strong passwords!
Continue reading

What Is Code?

I’ve been thinking recently about how some people have jobs that most everyone can “understand” more or less – we all know kinda sorta what a teacher or a policeman does – while others work in areas that are virtually inaccessible to the general public. I’m often reminded how little my family and closest friends understand about how I spend my days.

So Paul Ford of Businessweek has written this colossal, 38,000-word article “What Is Code” that attempts to bridge that gap. I’m not sure it succeeds (or that any article could) but it’s a fine attempt and a damn good read. Even for coders. It took hours to get through, and reading is not generally how I like to spend my weekend time, but it was time very well spent. Super recommend.

What Is Code