Django: Testing for Missing Migrations

When adding or altering model schemas in Django, developers typically generate and commit accompanying migration files. But, counterintuitively, Django wants to track all model changes in migration files, even if they don’t result in database schema changes. Those can be easy to miss.

Regardless the reason, you never want your repo to be in a state where migrations are detected as needed, but accompanying migration files aren’t committed.

Writing a pytest test that can be triggered on github, Circle, Travis, Jenkins, or whatever you use turns out to be trivial, but I couldn’t find documentation or examples on the interwebs, so am posting this here for posterity:

import pytest

from django.core.management import call_command

@pytest.mark.django_db
def test_for_missing_migrations():
    """ If no migrations are detected as needed, `result`
    will be `None`. In all other cases, the call will fail,
    alerting your team that someone is trying to make a
    change that requires a migration and that migration is
    absent.
    """

    result = call_command("makemigrations", check=True, dry_run=True)
    assert not result

It really is that simple! If no migrations are needed, result will be None. Any other return value means someone should run ./manage.py makemigrations to see what’s missing, and commit the results.

In Defense of Flickr

People like to say flip things like “Flickr is dead” just like they’ve been saying “Apple is dead” for decades. It’s true that Flickr’s heyday has passed, but there are still hundreds of thousands (millions?) of photographers posting there daily. Interaction is lively, there’s an interest group for every photography niche you can think of, and users are really supportive with the compliments and CC. Flickr still has upwards of 75 million accounts, and can receive up to 25 million photo uploads on a good day (stats).

Scientia, pulchritudo, lux

Flickr dominated the online photo sharing scene for around 15 years. But as Facebook’s popularity rose, FB became the world’s dominant photo sharing platform, eclipsing Flickr’s numbers (which were already astounding). After a recent brush with financial ruin, Flickr was purchased by SmugMug, and they’ve been great stewards of the platform so far, with improvements being released on the regular.

Sunset at Berkeley Marina

Somewhere in there, along came Instagram to soak up much of the remaining photographic juice in the room. Everyone is on it, so you’d be nuts not to use it, right? I use it too.

So why not just use Facebook and Instagram and call it a day? A bunch of reasons!:

  • Instagram images are tiny postage stamp versions of the images you’ve put so much work into. It’s almost an insult to your photos to display them so small with no full-size web option. But we all do it because we have to (and it’s fun).
  • Instagram has hashtags, but no real “groups” – no organized photo communities.
  • Facebook does full-screen, but they still compress and rewrite your images on upload, even if you enable the HD mobile setting.
  • Not everyone is on Facebook. Millions of people won’t use it for either personal or political reasons (I’m sure we’ve all seen numerous friends leave the platform in the past couple of years). People without FB accounts simply can’t see your work here. Out of bounds. Walled gardens have their place, but I don’t want my photos in one.
  • What if you decide to leave Facebook in a couple of years? What happens to all the work you posted here? Or will the fact that your photos are on FB prevent you from leaving the service even if you want to for other reasons? You’re “locked in.” Putting your images on Flickr instead means your images are decoupled from your social network, which gives you freedom.
  • Both Facebook and Instagram strip out all of your EXIF data, while Flickr does not. I really enjoy studying the EXIF data for other people’s images, or reminding myself of settings that were used on my own.
  • Only Flickr provides full and detailed statistics — not just of likes, but for all views, since the beginning of time (just realized my account is coming up on one million total views since I started there in 2005, wow!)

On Flickr, your images are available in super high resolution, with a wide variety of copyright options. There’s a huge number of interest groups, and detailed statistics. The challenge of getting an image into “Explore” (Flickr’s homepage featuring the best images on the service, changing constantly) is ongoing, and so rewarding when it happens. Discovering great new photographers daily is inspirational. Flickr is photo paradise to this day, IMHO.

Protester returning home, El Cerrito

All of which makes me wonder why so many photographers I meet aren’t using it. It seems like the best of all the options (yes there are other options, but they don’t compete with Flickr IMO, except maybe SmugMug (I’m curious about the features and community there too, honestly – promising).

Tower of Power

Are you on Flickr? And if not, why not?

FWIW #1: I usually post the same image to Instagram and Flickr at the same time, but only share the URL of the Flickr version, since it’s high-res. I only post images on Facebook occasionally, and just for the audience of my friends, rather than for the photo community.

FWIW #2: No I’m not a Flickr employee! Just a long-time user who never fell out of love.

FWIW #3: I’m shacker on Flickr and would love to follow you if you’re there too.

COVID-19 shopper at Trader Joe’s

Sync’ing Photos Between Computer and Phone (etc.)

Here are a few questions that come up over and over on photo forums. To me, these all amount to the same question. Or, at least they all have the same answer:

  • How can I post photos from my DSLR to Instagram (how to get DSLR images onto my phone?)
  • I have a computer, a phone, and a tablet. How can I transfer images between them?
  • If I delete or edit a photo on one device, how can I get those changes over to my other device?
  • “What do you all use for storage? An external hard drive?”

They’re good questions, and every new photographer has come up against them at some point. What surprises me is how many seasoned photographers still don’t have a good answer to them — surprising because I consider them a totally solved problem.

It’s 2020. All of your photos should live happily together in one library. That library should live in the cloud, so that when you add to or delete from or edit an image on one of your devices, that change is reflected on all of your devices. When you add an image from your DSLR to your photo collection, it should appear on your phone a few seconds later, ready to post to Instagram or whatever. You shouldn’t have to think about it – it should just work.

Equally important: No one should be having to keep photos in a bare folder somewhere, then drag them into Photoshop or Lightroom or whatever. All (pretty much all?) editing should happen directly in your Digital Asset Manager (DAM) – the tool you use to organize your photos into albums, and give them titles and descriptions and tags, to do all of your editing, organizing, sifting, sorting, importing and exporting to and from.

As far as I can tell, the best-integrated solution for this “one shared backing store” workflow is iCloud Photo Library from Apple. It works inside the Photos apps for MacOS and iOS, but changes the storage arrangement so that “master” copy of each image lives in the cloud, not on your hard drive (of course you must download the full-resolution master image to edit them, but that is a seamless process.

This makes some people feel nervous. We’re so used to controlling and backing up our own hard drives, it can be hard to hand over control. But think about it for a minute: How susceptible is your hard drive to fire, theft, or flood? How religious are you about backing it up? If you’re backing up off-site, what are you paying for that service? Do you really trust your little $100 hard drive and your own habits more than you trust a company holding the fate of hundreds of thousands of photographers’ work in their hands? If they made a mistake that bad, they’d be out of business. You can bet their backups are taken many times a day and distributed around the globe. And if you delete something by accident, you have a 30-day rolling Trash Can to fall back on. Seriously, I feel way safer with my photos in Apple’s hands than I did before.

At the risk of sounding like a fanboy, it’s hard to overstate just how liberating it is to have all of your synchronization issues just go away one day. The questions at the top of the page are just no longer questions. Use it for a few days and you’ll never want to go back.

$10/month, or $15/month for your whole family, is chicken scratch — you can hardly buy lunch for $10 these days. Great how-to guide here.

I jumped into iCloud Photo Library the day it was released and it solved all of the multi-camera / multi-computer problems I’d been banging my head against for years.

Other Options?

If you’re worried that the Photos app won’t be good enough to replace Lightroom, you might want to have another look at the the advanced editing tools in Photos — they’re extremely good! (I’d be curious to know whether there’s still anything you can do in Lightroom that you can’t do in the free Photos app?). Plus Photos supports plugins — I run Skylum Luminar not as a standalone app, but as a plugin for the Photos app. It’s mind-blowing what Luminar can do.

I believe that Windows users can also use iCloud Photo Library, but the integration workflow is a little less elegant. If you use Photo Library on Windows, please leave a comment and let us know what you think.

If you just feel super-married to Lightroom, the good news is that they now offer a cloud storage option as well, but I have no idea whether or how well it integrates with the Photos collection on your iOS or Android phone (if you know, let us know?)

Bottom line for me: Life is too short to be messing around with some of the convoluted workarounds people come up with to post their DSLR images on Instagram, or to deal with any of the other questions up top. You shouldn’t have to think about it – it should just work!

Oh my zsh

That version of bash we’ve been using on MacOS since forever hasn’t been updated since forever (for GPL licensing reasons). With Catalina, Apple finally got with the program and pushed bash into the background in favor of zsh. Nothing earth-shattering there – zsh has been available forever, but now it’s the default, and the Terminal will bug you on every launch if you don’t follow its instructions to migrate.

Once you do move over, run don’t walk to Oh My Zsh for a wonderful rabbit hole of themes, plugins, and configuration options to make zsh sing for you. By default, it knows how to detect whether the current directory contains a git checkout, and will show the active branch in the prompt, which is worth the price of admission alone.

New features in django-todo

If you haven’t checked out django-todo for a while, the project has been super-active lately! In the past few months it’s gained support for file attachments, batch-task-import via CSV, and a fully integrated email tracker. Now at version 2.4.6 on pipy, or check out the live demo site.

I started this project as a small demo more than ten years ago, and it’s evolved into a piece of staple software in the Django ecosystem. Proud of what it’s become!

TrackerBase

Around 20 years ago, enthralled by the database-like properties of the Be File System in BeOS, I decided to see if I could use the filesystem itself as a backing database for a live web server application. I ultimately wrote TrackerBase, and served betips.net from a BeBox in my home for a few years.

The basic idea was that BeOS let you create custom filetypes with custom attributes, and provided APIs for querying those attributes. The rest was just custom database query code, but without an actual database.

Fast forward two decades. I’m all-Mac now. I recently stumbled across the original .pkg installer for TrackerBase and wanted to have a look. Discovered that there was no way to open the package from the Mac side, and reached out to the Haiku community (Haiku is the open-source successor to BeOS). Within an hour, a community member provided me with a tarball, and I was able to see that code again for the first time in a long time.

It’s always embarrassing to look back on old code, but that was especially true in this case. And Perl? What was I thinking? Anyway, these days we have github, so I’ve just published the original code on github, untouched, for posterity’s sake. MIT license.

Irrelevant as it may be today, I’m still proud of it as  a novel application of an innovative filesystem.

Geo contribution to Python faker lib

Like most devs, I often have a need to generate tons of fake data to fill out models – addresses, usernames, “news” content, locations, IP addresses, you name it. Data-mocking libraries are invaluable for creating data factories for use in unit tests, and for populating a site under development with realistic content. For Python, I use the Faker lib in conjunction with FactoryBoy.

Recently I was working on a site that required generating a lot of maps, and realized that randomly chosen geographical coordinates were most often over ocean (since earth is mostly water). Realizing that the right way to solve this was to include a hash of locations known to exist on land and to then pull randomly from those, I started a pull request, which evolved over the past few weeks into a whole new geo module for the lib.

I started with the open source data sets at GeoNames, converted one of their databases into a Python tuple, then extracted every fifth entry to keep the dataset down to a manageable size. Pretty certain I’ve got every country on earth represented, but let me know if you find any missing.

I ended up bringing geographic features from other Faker modules into the new geo module, and added the ability to specify the country you want random land coords for. As of this morning pull request merged!

Crappy Mobile Photo Uploads to Facebook is the Default

Just discovered that if you upload a photo to Facebook from your phone, then upload the exact same image to the same post from desktop, the quality difference is huge. FB compresses the crap out of phone uploads.

Here’s proof — I uploaded these images from phone, didn’t give it much thought. Then later that day viewed the post on desktop/web and realized they looked like crap. My normal workflow is to do all posting from desktop, so I don’t typically see this effect, but this time, it was dramatic. I then added the same images to the post from desktop and arranged them side-by-side for easy comparison. FB doesn’t just compress a little from the phone, they compress like crazy.

Enough to get a feller running back to Flickr… but then I thought this must be an option or a setting. Went digging and sure enough… On the phone, go into Settings | Videos and Photos | Upload HD  to force the mobile app to upload your actual photos rather than compressed versions.

 

Interrobang

Whazzup‽ When “the interrobang” was introduced in 1962, virtually no typewriters or printing presses supported new glyphs, so it didn’t get traction. These days, almost everything you read supports a huge amount of glyph flexibility thanks to Unicode (emoji, right?).

Unfortunately, there’s no easy way to type an interrobang – unless you’re a super-nerd, you have to copy/paste them into your text. Same with a ton of other crazy glyphs that are probably supported by the font you’re using right now but not by the keyboard you’re typing on. IOTW we can see a lot more characters than we can type. You’re stuck (though fortunately you have easy access to the #octothorpe). Excellent episode of 99% Invisible:

Interrobang – 99% Invisible

Interrobang – 99% Invisible

In the beginning was the word, and the word was … well, actually, there was just one word … one long, endless word. For thousands of years, in some written languages, there was no space between words. People were expected figure out sentences and clauses while reading aloud. Scriptio continua was the dominant form of

Source: 99percentinvisible.org/episode/interrobang/

University CompSci Plagiarism

I’ve worked really hard on django-todo over the years, so was very dismayed to receive email recently from a CompSci student at the University of Western Australia informing me that the department had taken the django-todo source code, removed the license file and all attribution, and included its code in one of their assignments.

Of course I’m all for open source software being used for educational purposes, and would have been honored and pleased if they had done this in an OSS-respecting manner, but they chose to take the legally shady route instead, and to leave students with the impression that this was their original code. This is not how we educate students to become productive, respectful members of the open source software world.

The offending distribution is linked to from the CITS5501 Software Testing and Quality Assurance page – download the file cits5501-project-v2.zip.

I do hope the university takes some form of disciplinary action against the professor, who obviously knows better. And I hope the professor does the right thing. In case it’s not clear what “the right thing” is, I’ll spell it out: Restore the license file, and restore attribution from the files you removed it from. Better yet, save yourself a bunch of work and don’t include a copy of the software in your assignment download at all – just link to the github repo! That’s what it’s there for.