A Nanosecond Per Foot

It takes light takes about a nanosecond to travel a foot. So if you’re sitting four feet away from me, I’m seeing you as you were 4 billionths of a second ago. Meanwhile, Earth light takes four years to get to Alpha Centauri. So for anyone on a planet in Alpha Centauri watching Earth TV, it would appear that we still have an adult president in office.

Testing Reusable Django Apps with Pytest

Django project dependencies are normally installed with a simple

pip install myapp

But if you want to be a code contributor to a dependency such as a reusable Django app (or if you are writing one), the usual pattern is to:

  1. git clone that app to another directory, outside of your project
  2. From within your project dir, use
    pip install -e /path/to/the/clone

This creates a foo.pth link and/or a foo.egg-link in your project’s virtualenv, rather than copying the code into the project itself, so you can work on the separate repo and have changes reflected in your project in real time.

That’s all well and good, but if you use pytest and need to run tests in the dependency, just running pytest from your project dir won’t find tests that live in the dependency. So can’t you just run pytest from within the dependency’s directory? Not quite – since it’s a Django app, it’s going to assume access to Django itself, and likely to other apps installed in the Django project, such as users. It would be redundant and cumbersome to install Django in the dependency’s directory, and that still wouldn’t solve the problem of access to users etc.

I can’t seem to find mention anywhere of a “best-practice” solution to this dilemma, but after much futzing around and feeling like I must be missing something obvious I tried what should have been plain from the outset:

  1. From the project dir, pip uninstall myapp
  2. From the project dir, create a symlink:
    ln -s /path/to/the/clone myapp
  3. Add myapp to your .gitignore

Now the separate reusable app appears to Django and to pytest like it’s present in the project, while maintaining physical separation between the project and the dependency. Tests run, and there’s no confusion. Just be sure to .gitignore the symlink.

There’s a gotcha to look out for here: Typically, a reusable app called “myapp” will live in a parent folder that’s also called “myapp”. If you make your symlink to the top-level folder, the imports inside it will not work. You’ll want to create your symlink to the inner folder, e.g.:

ln -s /Users/yourname/dev/myapp/myapp .

Unless I’m mistaken about something, a simple symlink (rather than pip install -e appears to be the easiest way to organize the relationship.

How To Screw Up a Good Backup System

I back up our family computers like crazy, using a combination of Time Machine and cloud backup via Backblaze. But I did something dumb and almost lost our family’s entire history of home videos. Facing estimates of $500 – $1500 for professional data recovery, I stumbled on an awesome hack that saved the day.

Background

All of our computers’ internal hard drives get a dedicated external Time Machine, and we use Backblaze for extra insurance, so our data is safe in the cloud in case of fire, theft or flood. But we also have a few external hard drives that store things like large music collections and our home videos. The external drives back up to Backblaze only (no Time Machine).

All of that has been working hunky dory for years, and I felt confident we were safe. Then, a week ago, I realized that the drive that stores our family videos (“Gorgonzola”) would no longer mount, with any cable, on any of our machines. Yikes! So I turned to Backblaze for a restore, only to find it wasn’t showing up there either! Double yikes, freakout.

What Happened

At some point in the distant past, I did something dumb, though I didn’t realize it was dumb at the time – I realized I had lots of extra space on Gorgonzola and decided to let that drive do double-duty, as a Time Machine drive for a laptop. What I didn’t take into account was the fact that Backblaze has a reasonable rule – they don’t back up your backups. So when Backblaze detected that Gorgonzola was now a Time Machine drive, it dropped it from the manifest. I never noticed it had been dropped.

So now I’d realized that I had NO backup available anywhere for this precious, unreplaceable data. So I called some data recovery services, and got estimates ranging from $500 – $1500. The data was important enough to me that I’d pay that ransom, if it came to it. But of course I didn’t want to.

So Crazy It Just Might Work

The next day, I stumbled on a brilliant suggestion: Often, when a drive won’t mount, it’s because the USB controller circuit board inside the drive case has gone south, and that the drive itself is fine. Solution: Purchase an identical drive, take them both apart, and swap the controller chips. Brilliant! Found an identical drive on Amazon for $100 (Seagate Backup Plus Slim), and went for it.

Prying the case open and removing the drives turned out to be easier than expected (YouTube video), and the controller chips slipped off easily. Quick swaperoo, and lo and behold, it worked! Gorgonzola showed up as normal, and I’m rescuing my data right now. Of course, both drives were destroyed in the process, but at this point, I don’t care.

Yay, internet.

Django template tag to display latest git commit, date and tag in template

I found a few references to bits and pieces of this in various places, but this snippet ties all three bits of information together in a single string, and accounts for calling the path to the git repo from anywhere (which tends to work “magically” on localhost but breaks on servers).

If you don’t use git tags, season to taste.

@register.simple_tag
def git_ver():
    '''
    Retrieve and return the latest git commit hash ID and tag as a dict.
    '''

    git_dir = os.path.dirname(settings.BASE_DIR)

    try:
        # Date and hash ID
        head = subprocess.Popen(
            "git -C {dir} log -1 --pretty=format:\"%h on %cd\" --date=short".format(dir=git_dir),
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        version = head.stdout.readline().strip().decode('utf-8')

        # Latest tag
        head = subprocess.Popen(
            "git -C {dir} describe --tags $(git -C {dir} rev-list --tags --max-count=1)".format(dir=git_dir),
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        latest_tag = head.stdout.readline().strip().decode('utf-8')

        git_string = "{v}, {t}".format(v=version, t=latest_tag)
    except:
        git_string = u'unknown'

    return git_string

Then, in your template, you can simply:

{% if user.is_superuser %}{% git_ver %}{% endif %}

GoPro Time Lapse: Stinson Beach

Last weekend at Stinson Beach I attached a GoPro to a tree and had it shoot one image every 5 seconds for a couple of hours. Later compiled the image into a 29fps half-speed video in GoPro Studio. Not sure why I enjoy making these so much; something oddly satisfying about the process.

View 403, 404, 500 with media in Django DEBUG mode

When working with Django in DEBUG mode, it can be tough to see your 403, 404, and 500 views, since they raise visible stack traces instead of the UX the end user will see. But if you turn DEBUG off, runserver’s local media serving is disabled because it’s designed to work only with DEBUG = True. The solution is scattered throughout the Django docs, and I couldn’t find it compiled into one compact code block anywhere – just reference the handling functions directly from the end of your urls.py:

if settings.DEBUG:
    from django.views.defaults import server_error, page_not_found, permission_denied
    urlpatterns += [
        url(r'^500/$', server_error),
        url(r'^403/$', permission_denied, kwargs={'exception': Exception("Permission Denied")}),
        url(r'^404/$', page_not_found, kwargs={'exception': Exception("Page not Found")}),
    ]

And voila, your 403.html, 404.html, and 500.html templates will be displayed in full glory for developers.

Remove Duplicates, Fix Broken Album Art in iCloud Music Library

Apple Music + iCloud Music Library is a brilliant pairing, and finally lets us access our personal music collections from anywhere. But it’s not without its warts – duplicated tracks and bad/missing cover art has been a sore spot for iCloud Music Library users since the service launched. In my first piece for Medium.com, I walk readers through the reasons – and the fixes – for those two problems.

record-collection

The Ethics of Modern Web Ad-Blocking

For years, I’ve resisted – and argued against – using web ad blockers of any kind. After spending a decade working at a journalism school and watching publishers large and small struggle (and mostly fail) to find a way to be paid for their essential efforts, it felt like bearing a certain amount of advertising was the very least we-the-public could do to support quality journalism. Paywalls don’t work for almost any publications – what else is there?

But the rise of the mobile web tipped the scales – the “social contract” around advertising was no longer a fair one. The mobile experience is far less tolerant of intrusion, and network speeds are slower. But because monetization is more difficult, publishers were “forced” to insert more, and more intrusive advertising. The cumulative effect has been a steady decline in the quality of mobile browsing. Today, many news sites are close to unusable on a smart phone, having become choked out by network and screen-stealing crap.
Continue reading