Software Is Hard

Andrew Leonard reviews Salon co-founder Scott Rosenberg’s new book, “Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software. For several years, Rosenberg followed the (now largely stalled) development process of the open source PIM Chandler, and was also dragged through the development process for’s custom content management system (a process I myself went through last summer). On why Vista and other large development projects are almost predictably over-budget and very late:

… the incredible difficulty of estimating the time it takes to do this stuff, whether you are building a little content management system for a relatively modest-size Web company or whether you are building the operating system that will be used by three-fourths of the known universe. The difficulty in saying, A) How long will it take to do what you want to do? And B) When are you done doing what you want to do?

The answer, it turns out, is incredibly elusive, and very few people are capable of estimating these things with anything approaching precision. On build vs. buy:

And the programmer who says “it will be faster for me to write it, rather than to learn it,” is usually correct. Except that what he will write, most likely, is something that will work but will not have its rough edges worked out.

Touche’. I’m becoming convinced that the ideal middle-ground on the build vs. buy spectrum is to use a development framework like Rails or Django. Then you aren’t building all of the plumbing yourself, and you still get to lean on the hard work of others, but you also aren’t constrained by the models and methods of fully-baked systems (like pre-rolled content management systems). The downside to frameworks is that, to be productive, you need more up-front training than you would when building from scratch, or when using an existing product. To that end, I’ll be doing another week of pure research/training next week – this time on Python/Django (after I get some molars removed, that is). Whether a week will be enough to start feeling productive with it is something I’ll soon find out.

Music: Arbouretum :: Sleep Of Shiloam

16 Replies to “Software Is Hard”

  1. Scot,

    Why Python/Django and not Ruby on Rails? Just curious…

    I started playing with RoR a couple of weeks ago, but I rarely have time to fully focus on it… I’m up to page 94 (probably after two _real_ evenings?) in the Pragmatic Programmer book on rails and it is very good. I never get through books quickly, but this one is clear enough to make reading (and trying) go quickly.

    I looked at Symphony (PHP) for a while but there was just a little too much configuration. Code Igniter is a nice bare-bones framework for PHP, too.

    I didn’t really look at Python and Django, but I know the Washington Post is doing some really neat things with Django.

    One of the two individuals that is programming the project is a huge Django fan.

  2. Gilbert – Glad you asked. Truth be known, I actually advocated for RoR or a PHP-based framework over Django. RoR because of the momentum it has over Django, and PHP-based frameworks so we could leverage our existing skills. But none of the existing PHP frameworks have the level of buy-in that Django and RoR are enjoying, and I don’t think they’re quite as mature. I played with Symfony for half a day, and couldn’t even get the tutorials to work as advertised (which probably says more about documentation writing in the open source world than it does about Symfony, but it made for a poor introduction).

    The reasons for adopting Django were two-fold: 1) Because we’re a journalism school and Django is seeing a strong uptick in adoption in the online news site world. Of course there’s absolutely nothing about Django that makes it better or worse suited to news delivery or multimedia than any other framework, but powers-that-be wanted to tap into that buzz; and 2) The other developer I work with researched RoR and Django and found more praise for Django as a pure framework than for RoR. I read some of the arguments and some of them sounded compelling. For example, I hate that RoR can’t run cleanly through mod_ruby – you have to do all this stupid CGI redirect stuff in Apache to deploy. Django runs through mod_python, which is much cleaner. Another biggy is the reusabilty in Django. Build some component of a web app for one site and you can re-use it in another app on the same server with no extra work. You don’t copy it over and modify it – you just locate it centrally so both apps can use it. Object oriented down to the most basic level. Also, the Admin interface you get for free with any Django project goes way beyond what you get with Rails.

  3. I thought I wrote a reply to this… I guess I forgot to prove that I’m not a computer ;)

    I don’t remember exactly what I said, but the gist of it was that I really like the things that you pointed out about Django (better free admin, mod_python, oo through-and-through). Once I’ve gotten a reasonable ROI on my RoR books, I’ll be taking a look at Django. I’ve had my fairshare of RoR apps gone bad on shared servers over at Textdrive to know that it IS a problem. It looks like they have a handle on the problem now, but for a LONG time is was unbearable.

  4. Interesting. And I tried really hard (as in, really hard) to set up RoR as an option for Birdhouse customers. But it was just such a pain in the neck to get the funky Apache integration set up in a way that would be user-friendly that I finally gave up. We actually do have all the bits and pieces set up, but I don’t advertise it, and users are on their own in terms of integration. It really seems like the RoR community would have gotten this one solved a long time ago — it’s a major achille’s heel — enough to drive many back to PHP.

  5. Ugly, oh yeah. For the gruesome details see here:

    I’ve never used RoR over there, I just had other people using RoR ruining my experience with PHP/MySQL apps.

    I don’t know if the RoR community is looking at these problems, but they should be.

  6. Gilbert – Looks like you found your way into Akismet somehow. I unspammed your comment. I was envious of TextDrive for a while – now I feel a bit sorry for them – they offered mass support for what I feel is an undercooked technology, and ended up with a big mess on their hands.

    Xian – I doubt it. They’ve just poured all of their effort into the core technology and too little into deployment.

  7. You’re killing me, Scot. My A.D.D. is kicking in… must. stay. focused. on. learning. ruby. and. rails…

  8. Ok, I spent some of Saturday installing PostgreSQL and Django. Then, most of Sunday trying to get Django to work. I couldn’t get past chapter 3 of the Django book. My dynamic content view and URLconf refused to play nice with one another — spitting errors. I didn’t even get a chance to try the PostgreSQL install!

    I think I was caught in some sort of versioning hell. I don’t know if it is my Python version or Django version that is throwing me off.

    Anyhow, I think I’ve satisfied my A.D.D. to try Django, for now, back to RoR! I’ll come back to try again when Django has hit 1.0 and the Django book in synced with the publically released version instead of the SVN version.

  9. Hmm… you should be able to work with versions of Python back to 2.3. Not sure what you’re experiencing there, but feel free to send me your code and I’ll see if I can figure out what’s going on. Though I understand if you don’t feel like your distracting yourself. It’s hard enough to learn one framework, without muddying the waters.

    I know how it feels to have a tutorial not work out. Sucks, basically. Enough to turn one away for good, however unfair that may be.

    Note the difference between the Django book, which does require the latest Django, and the tutorials, which work with older versions of Django.

  10. Thanks for the offer, Scot. However, I think I worked the “what am I missing fever” over enough to refocus on RoR, which is working better for me so far.

    I WILL try Django at another time. I haven’t given up completely, bad tutorial experience or not.

    > Note the difference between the Django book, which does require the latest Django, and the tutorials, which work with older versions of Django.

    Yes, I think this is what tripped me up. I thought the latest “release” was good enough, but I think they want you to check out the SVN version.

  11. I once met the gDjango fan doing the stuff for the Washington Post, Adrian Holovaty, He is sharp as a whip and extremely friendly and helpful — I imagine he would welcome you reaching out to him if needed.

Leave a Reply

Your email address will not be published. Required fields are marked *