Django template tag to display latest git commit, date and tag in template

I found a few references to bits and pieces of this in various places, but this snippet ties all three bits of information together in a single string, and accounts for calling the path to the git repo from anywhere (which tends to work “magically” on localhost but breaks on servers).

If you don’t use git tags, season to taste.

def git_ver():
    Retrieve and return the latest git commit hash ID and tag as a dict.

    git_dir = os.path.dirname(settings.BASE_DIR)

        # Date and hash ID
        head = subprocess.Popen(
            "git -C {dir} log -1 --pretty=format:\"%h on %cd\" --date=short".format(dir=git_dir),
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        version = head.stdout.readline().strip().decode('utf-8')

        # Latest tag
        head = subprocess.Popen(
            "git -C {dir} describe --tags $(git -C {dir} rev-list --tags --max-count=1)".format(dir=git_dir),
            shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
        latest_tag = head.stdout.readline().strip().decode('utf-8')

        git_string = "{v}, {t}".format(v=version, t=latest_tag)
        git_string = u'unknown'

    return git_string

Then, in your template, you can simply:

{% if user.is_superuser %}{% git_ver %}{% endif %}

View 403, 404, 500 with media in Django DEBUG mode

When working with Django in DEBUG mode, it can be tough to see your 403, 404, and 500 views, since they raise visible stack traces instead of the UX the end user will see. But if you turn DEBUG off, runserver’s local media serving is disabled because it’s designed to work only with DEBUG = True. The solution is scattered throughout the Django docs, and I couldn’t find it compiled into one compact code block anywhere – just reference the handling functions directly from the end of your

if settings.DEBUG:
    from django.views.defaults import server_error, page_not_found, permission_denied
    urlpatterns += [
        url(r'^500/$', server_error),
        url(r'^403/$', permission_denied, kwargs={'exception': Exception("Permission Denied")}),
        url(r'^404/$', page_not_found, kwargs={'exception': Exception("Page not Found")}),

And voila, your 403.html, 404.html, and 500.html templates will be displayed in full glory for developers.

Sane Password Strength Validation for Django with zxcvbn

While many admins and blog posts tell users that length is by far the most important factor in creating strong passwords/passphrases, the majority of password input fields are giving them a set of hide-bound rules: Eight characters, at least one upper- and one lowercase letter, some digits and punctuation marks, etc.

Even though it includes dictionary words, a passphrase like:

Sgt. Pepper's Mr. Kite

is far stronger than:


(there’s a world of difference between 22 characters and 9, from a cracking perspective). But many password input fields would reject the first one. No wonder users are confused by the process of creating strong passwords!
Continue reading

Django Unit Tests Against Unmanaged Databases

A Django project I’m working on defines two databases in its config: The standard/default internal db as well as a remote legacy read-only database belonging to my organization. Models for the read-only db were generated by inspectdb, and naturally have managed = False in their Meta class, which prevents Django from attempting any form of migration on them.

Unfortunately, that also prevents the Django test runner from trying to create a schema mirror of it during test runs. But what if you want to stub out some sample data from the read-only database into a fixture that can be loaded and accessed during unit tests? You’ll need to do the following:

  • Tell Django to create the second test database locally rather than on the remote host
  • Disable any routers you have that route queries for certain models through the remote db
  • Tell Django to override the Managed = False attribute in the Meta class during the test run

Putting that all together turned out to be a bit tricky, but it’s not bad once you understand how and why you need to take these steps. Because you’ll need to override a few settings during test runs only, it makes sense to create a separate to keep everything together:

from project.local_settings import *
from django.test.runner import DiscoverRunner

class UnManagedModelTestRunner(DiscoverRunner):
    Test runner that automatically makes all unmanaged models in your Django
    project managed for the duration of the test run.
    Many thanks to the Caktus Group:

    def setup_test_environment(self, *args, **kwargs):
        from django.db.models.loading import get_models
        self.unmanaged_models = [m for m in get_models() if not m._meta.managed]
        for m in self.unmanaged_models:
            m._meta.managed = True
        super(UnManagedModelTestRunner, self).setup_test_environment(*args, **kwargs)

    def teardown_test_environment(self, *args, **kwargs):
        super(UnManagedModelTestRunner, self).teardown_test_environment(*args, **kwargs)
        # reset unmanaged models
        for m in self.unmanaged_models:
            m._meta.managed = False

# Since we can't create a test db on the read-only host, and we
# want our test dbs created with postgres rather than the default, override
# some of the global db settings, only to be in effect when "test" is present
# in the command line arguments:

if 'test' in sys.argv or 'test_coverage' in sys.argv:  # Covers regular testing and django-coverage

    DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
    DATABASES['default']['HOST'] = ''
    DATABASES['default']['USER'] = 'username'
    DATABASES['default']['PASSWORD'] = 'secret'

    DATABASES['tmi']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
    DATABASES['tmi']['HOST'] = ''
    DATABASES['tmi']['USER'] = 'username'
    DATABASES['tmi']['PASSWORD'] = 'secret'

# The custom routers we're using to route certain ORM queries
# to the remote host conflict with our overridden db settings.
# Set DATABASE_ROUTERS to an empty list to return to the defaults
# during the test run.


# Set Django's test runner to the custom class defined above
TEST_RUNNER = 'project.test_settings.UnManagedModelTestRunner'

With that in place, you can now run your tests with:

./ test --settings=project.test_settings

… leaving settings untouched during normal site operations. You can now serialize some data from your read-only host and load it as a fixture in your tests:

class DirappTests(TestCase):

    # Load test data into both dbs:
    fixtures = ['auth_group.json', 'sample_people.json']


    def test_stub_data(self):
        # Guarantees that our sample data is being loaded in the test suite
        person = Foo.objects.get(id=7000533)
        self.assertEqual(person.first_name, "Quillen")

Displaying Django User Messages with Angular.js

Django’s Messages framework is an elegant workhorse, and I’ve never built a Django site that didn’t use it for displaying success/failure/info messages to users after certain actions are taken (like logging in successfully or adding an item to a cart).

But wouldn’t it be cool if you could use that functionality client-side, delivering user messages to be processed as JSON data rather than statically outputting messages to generated HTML? On a recent project, I needed to do this because Varnish caching doesn’t let you mark page fragments as non-cacheable, so statically generated messages were not an option. But there are all sorts of reasons you might want to handle Django Messages client-side.


Here’s how to accomplish the job in a really lightweight way, without the need for a full-blown REST API app like Django Rest Framework or Tastypie, and with Angular.js (which is, IMO, the best of the current crop of JavaScript application frameworks).
Continue reading

django-allauth: Retrieve First/Last Names from FB, Twitter, Google

Of the several libraries/packages available for setting up social network logins for Django projects, I currently find django-allauth the most complete, with the best docs and the most active development. Doesn’t hurt that the lead dev on the project is super friendly and responsive on StackOverflow!

But not everything about it is intuitive. After wiring up Twitter, Facebook and Google as login providers, I found that first and last names were not being retrieved from the remote services when an account was successfully created. I also, frustratingly, could find only the most oblique references online to how to accomplish this.

There are a couple of ways to go about it – you can either receive and handle the allauth.account.signals.user_signed_up signal that allauth emits on success, or set up allauth.socialaccount.adapter.DefaultSocialAccountAdapter, which is also unfortunately barely documented.

I decided to go the signals route. The key to making this work is in intercepting the sociallogin parameter your signal handler will receive when an account is successfully created. I then installed a breakpoint with import pdb; pdb.set_trace() to inspect the contents of sociallogin. Once I had access to those goodies, I was able to post-populate the corresponding User objects in the database.

This sample code grabs First/Last names from Twitter, Facebook or Google; season to taste:

# When account is created via social, fire django-allauth signal to populate Django User record.
from allauth.account.signals import user_signed_up
from django.dispatch import receiver

def user_signed_up_(request, user, sociallogin=None, **kwargs):
    When a social account is created successfully and this signal is received,
    django-allauth passes in the sociallogin param, giving access to metadata on the remote account, e.g.:

    sociallogin.account.provider  # e.g. 'twitter' 

    See the socialaccount_socialaccount table for more in the 'extra_data' field.

    if sociallogin:
        # Extract first / last names from social nets and store on User record
        if sociallogin.account.provider == 'twitter':
            name = sociallogin.account.extra_data['name']
            user.first_name = name.split()[0]
            user.last_name = name.split()[1]

        if sociallogin.account.provider == 'facebook':
            user.first_name = sociallogin.account.extra_data['first_name']
            user.last_name = sociallogin.account.extra_data['last_name']

        if sociallogin.account.provider == 'google':
            user.first_name = sociallogin.account.extra_data['given_name']
            user.last_name = sociallogin.account.extra_data['family_name']

Center for Investigative Reporting

cir A year and a half ago, I left the Berkeley J-School to experience life in a high-energy web development shop with central campus. I learned a ton in that short time – the Agile process, Angular.js, building sites as Single Page Applications, strict separation between back-end and front-end systems, rigorous code review processes, and much more. And I had the opportunity to work with a crew of Java, Ruby, and Javascript rock stars, from whom I’ve learned so much.

Since my career to date had been as a web tech generalist (i.e. one person wearing all the hats), I found the experience incredibly illuminating. And yet… the project and I had some “creative differences” which ultimately resulted in me leaving the department at the end of May.

I’ve spent the past month working on personal and freelance projects, studying, and job hunting. I longed to work with journalists again, and really missed working with Django, which still feels like the most natural and effective way to build highly customized data-driven web sites I’ve ever encountered. At the same time, I wanted to make sure that my work had some kind of higher purpose – I wanted to be part of something with social and political impact.

Unfortunately, I wasn’t able to find anything on campus that really fit my requirements, and finally made the tough decision to start looking off-campus.

Today, I’m thrilled to say that I believe I’ve found the perfect fit, as a full-time Django developer at the Center for Investigative Reporting in Berkeley.

At The Center for Investigative Reporting (CIR), we believe journalism that moves citizens to action is an essential pillar of democracy. Since 1977, CIR has relentlessly pursued and revealed injustices that otherwise would remain hidden from the public eye. Today, we’re upholding this legacy and looking forward, working at the forefront of journalistic innovation to produce important stories that make a difference and engage you, our audience, across the aisle, coast to coast and worldwide.

CIR recently merged with the Bay Citizen and California Watch, two excellent journalism organizations that have had myriad overlapping projects with the J-School over the years. In fact, walking around the CIR offices today, I’m meeting former J-School students and instructors I haven’t seen in years – kind of a homecoming!

I’ll be enthusiastically┬ástarting work in mid-July. Yes, it’s tough to say goodbye to the University, but it really is an ideal evolutionary step for me right now.