Python For "One-Hour" Scripts

I wrote a "quick" script at work yesterday. I always have to put "quick" in quotes when I do this. I try to be good and write code to automate repetitive tasks that I do. Many others have advocated this and I find it to be very good advice, especially if you are a programmer. For me it usually happens like this: I'm doing something repetitive or tedious and I think, hey, I could spend an hour and write a script to do this. Over my career I've used Perl, shell scripts, and Python to do this. In that order. Yes, I learned Perl before I learned shell scripting (bash and korn shell on HP-UX), and no, Python hasn't completely replaced shell scripting, but I've all but abandoned Perl (and for my real firmware work I program in C++, just so you know).

Whenever this happens it always takes more than an hour to write one of these "one-hour scipts." Yes, I'll confess. Sometimes much longer. With Perl most tasks would only require a few lines of code that I could write in 15 minutes, and those few lines would be syntactically correct and would do close to what I wanted. Then it would always take me about three more hours to get them to do exactly what I wanted, usually because of some idiomatic Perl subtlety. Modifying or even just reading scripts later was painful too. It was pretty frustrating but I learned a lot and I did improve, but not by a whole lot. Shell scripts take me longer to get code that is actually syntactically correct and runs, but once I get there it is usually doing what I want. I don't write enough of shell scripts to memorize when you use brackets after an if, and just what the spacing should be around them and the stuff inside, where do I need a semi-colon, what grep or sed options do I need, and so forth. They syntax for shell scripts (and chaining the various UINX tools together inside said scripts) is really hostile. One-hour shell scripts take about as long as one-hour Perl scripts did. Reading them later isn't as bad a Perl.

Python, on the other hand is really, really nice. The syntax is so clean and the built-in libraries and functions are so handy. Even when months go by without me writing one of my "one-hour scripts" I can get back into Python easily. I can even modify and re-use methods from other scripts I've written without having to pour over them trying to figure out what the code is doing. I think I probably could even finish those one-hour scripts in an hour, but it's funny, I usually still take as much time as I used to with Perl or Bash. I'm usually spending the extra time, and this is embarrassing, cleaning up the script and making it "production" worthy, or just trying out new things. You know, refactoring, adding command-line options with optparse, adding docstrings so I can run pydoc on my script and have it produce pretty documentation, generalizing and moving code into modules that other one-hour scripts can import, replacing for loops with list comprehensions or generator expressions, stuff like that. It's downright ridiculous to do for these one-off throw-away one-hour scripts, but it's good practice, right? It's way more fun that debugging Perl or Bash was, and I'm learning stuff, and it's still within the "one-hour script" time budget, so it's all good.

The funny thing is, writing Python has helped my C++ coding too. Object oriented or functional programming techniques that are possible in C++ are drop-dead simple in Python. I've been able to easily try out and get comfortable with the concepts in these little Python scripts first, and then go tackle the same concept in C++ with it's much more cumbersome syntax. I may be the only firmware engineer who has written my own functor or used a back_inserter. How cool is that? (no really, I'm not sure...is it cool if nobody else can understand my code? Hmmm...)

In the end, I know, this is probably old news for many of you, but Python rocks. Go spend an hour and automate something tedious with it.