Django Unit Tests Against Unmanaged Databases

A Django project I’m working on defines two databases in its config: The standard/default internal db as well as a remote legacy read-only database belonging to my organization. Models for the read-only db were generated by inspectdb, and naturally have managed = False in their Meta class, which prevents Django from attempting any form of migration on them.

Unfortunately, that also prevents the Django test runner from trying to create a schema mirror of it during test runs. But what if you want to stub out some sample data from the read-only database into a fixture that can be loaded and accessed during unit tests? You’ll need to do the following:

  • Tell Django to create the second test database locally rather than on the remote host
  • Disable any routers you have that route queries for certain models through the remote db
  • Tell Django to override the Managed = False attribute in the Meta class during the test run

Putting that all together turned out to be a bit tricky, but it’s not bad once you understand how and why you need to take these steps. Because you’ll need to override a few settings during test runs only, it makes sense to create a separate test_settings.py to keep everything together:

from project.local_settings import *
from django.test.runner import DiscoverRunner


class UnManagedModelTestRunner(DiscoverRunner):
    '''
    Test runner that automatically makes all unmanaged models in your Django
    project managed for the duration of the test run.
    Many thanks to the Caktus Group: http://bit.ly/1N8TcHW
    '''

    def setup_test_environment(self, *args, **kwargs):
        from django.db.models.loading import get_models
        self.unmanaged_models = [m for m in get_models() if not m._meta.managed]
        for m in self.unmanaged_models:
            m._meta.managed = True
        super(UnManagedModelTestRunner, self).setup_test_environment(*args, **kwargs)

    def teardown_test_environment(self, *args, **kwargs):
        super(UnManagedModelTestRunner, self).teardown_test_environment(*args, **kwargs)
        # reset unmanaged models
        for m in self.unmanaged_models:
            m._meta.managed = False

# Since we can't create a test db on the read-only host, and we
# want our test dbs created with postgres rather than the default, override
# some of the global db settings, only to be in effect when "test" is present
# in the command line arguments:

if 'test' in sys.argv or 'test_coverage' in sys.argv:  # Covers regular testing and django-coverage

    DATABASES['default']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
    DATABASES['default']['HOST'] = '127.0.0.1'
    DATABASES['default']['USER'] = 'username'
    DATABASES['default']['PASSWORD'] = 'secret'

    DATABASES['tmi']['ENGINE'] = 'django.db.backends.postgresql_psycopg2'
    DATABASES['tmi']['HOST'] = '127.0.0.1'
    DATABASES['tmi']['USER'] = 'username'
    DATABASES['tmi']['PASSWORD'] = 'secret'


# The custom routers we're using to route certain ORM queries
# to the remote host conflict with our overridden db settings.
# Set DATABASE_ROUTERS to an empty list to return to the defaults
# during the test run.

DATABASE_ROUTERS = []

# Set Django's test runner to the custom class defined above
TEST_RUNNER = 'project.test_settings.UnManagedModelTestRunner'

With that in place, you can now run your tests with:

./manage.py test --settings=project.test_settings

… leaving settings untouched during normal site operations. You can now serialize some data from your read-only host and load it as a fixture in your tests:

class DirappTests(TestCase):

    # Load test data into both dbs:
    fixtures = ['auth_group.json', 'sample_people.json']

    ...

    def test_stub_data(self):
        # Guarantees that our sample data is being loaded in the test suite
        person = Foo.objects.get(id=7000533)
        self.assertEqual(person.first_name, "Quillen")

10 Replies to “Django Unit Tests Against Unmanaged Databases”

  1. HI where does this file should go? the app folder or project folder?
    in line 1 ” from project.local_settings import * ” , Do I have to change project to my project name and local_settings to my project settings file?
    Thank you in advance!!

  2. Julio, you would put it in the same dir with your other settings files. Yes, you would certainly tweak any vars or names to match your own project.

  3. Hi Shacker.
    Ok this is my file structure.

    manage.py
    tothem
    settings.py
    urls.py
    wsgi.py
    log_in
    migrations
    statics
    templates
    admin.py
    models.py
    test.py
    urls.py
    views.py

    so, I have to put “test_settings.py” inside tothem?
    line 1 should be “from settings import *” ?
    and I run the test like “./manage.py test –settings=tothem.test_settings”

    Thank you for the support!!!!

  4. Julio, I can’t answer all of these questions for you – file under “season to taste,” so you should set up however you like. I am demonstrating a solution to testing with unmanaged databases, not giving advice on how to configure your project.

  5. FYI for others who found this and then were disappointed it no longer worked: In Django 1.7+, test databases are created from migrations, not directly from models, so changing the current managed state of a model from False to True at runtime won’t make a difference. However, you can use the django-test-without-migrations pypi package with this method to create test tables from models. w00t!

    https://pypi.python.org/pypi/django-test-without-migrations/

  6. Update for django 1.9: After hours of messing with various ideas on the internet I found a solution.

    add the test-without-migrations module to the installed apps as Sarah Fowler suggested. Then instead of messing around with the test runner just search and replace the models.py file and update the managed = False to be managed = True
    Note: Your models.py file might have a diff regex

    create a bash script script (attempting markdown syntax…)
    “`
    #!/bin/bash
    sed -i ‘s/managed\s=\sFalse/managed = True/g’ admins/models.py
    python manage.py test –nomigrations
    sed -i ‘s/managed\s=\sTrue/managed = False/g’ admins/models.py
    “`

    Then run that bash script and everything should be good :)

  7. Hi,

    django.db.models.loading was removed from Django 1.9

    Use this one instead:

    from django.apps import apps
    self.unmanaged_models = [m for m in apps.get_models() if not m._meta.managed]

    Regards,
    Pablo

Leave a Reply

Your email address will not be published.