Python For "One-Hour" Scripts

I wrote a "quick" script at work yesterday. I always have to put "quick" in quotes when I do this. I try to be good and write code to automate repetitive tasks that I do. Many others have advocated this and I find it to be very good advice, especially if you are a programmer. For me it usually happens like this: I'm doing something repetitive or tedious and I think, hey, I could spend an hour and write a script to do this. Over my career I've used Perl, shell scripts, and Python to do this. In that order. Yes, I learned Perl before I learned shell scripting (bash and korn shell on HP-UX), and no, Python hasn't completely replaced shell scripting, but I've all but abandoned Perl (and for my real firmware work I program in C++, just so you know).

Whenever this happens it always takes more than an hour to write one of these "one-hour scipts." Yes, I'll confess. Sometimes much longer. With Perl most tasks would only require a few lines of code that I could write in 15 minutes, and those few lines would be syntactically correct and would do close to what I wanted. Then it would always take me about three more hours to get them to do exactly what I wanted, usually because of some idiomatic Perl subtlety. Modifying or even just reading scripts later was painful too. It was pretty frustrating but I learned a lot and I did improve, but not by a whole lot. Shell scripts take me longer to get code that is actually syntactically correct and runs, but once I get there it is usually doing what I want. I don't write enough of shell scripts to memorize when you use brackets after an if, and just what the spacing should be around them and the stuff inside, where do I need a semi-colon, what grep or sed options do I need, and so forth. They syntax for shell scripts (and chaining the various UINX tools together inside said scripts) is really hostile. One-hour shell scripts take about as long as one-hour Perl scripts did. Reading them later isn't as bad a Perl.

Python, on the other hand is really, really nice. The syntax is so clean and the built-in libraries and functions are so handy. Even when months go by without me writing one of my "one-hour scripts" I can get back into Python easily. I can even modify and re-use methods from other scripts I've written without having to pour over them trying to figure out what the code is doing. I think I probably could even finish those one-hour scripts in an hour, but it's funny, I usually still take as much time as I used to with Perl or Bash. I'm usually spending the extra time, and this is embarrassing, cleaning up the script and making it "production" worthy, or just trying out new things. You know, refactoring, adding command-line options with optparse, adding docstrings so I can run pydoc on my script and have it produce pretty documentation, generalizing and moving code into modules that other one-hour scripts can import, replacing for loops with list comprehensions or generator expressions, stuff like that. It's downright ridiculous to do for these one-off throw-away one-hour scripts, but it's good practice, right? It's way more fun that debugging Perl or Bash was, and I'm learning stuff, and it's still within the "one-hour script" time budget, so it's all good.

The funny thing is, writing Python has helped my C++ coding too. Object oriented or functional programming techniques that are possible in C++ are drop-dead simple in Python. I've been able to easily try out and get comfortable with the concepts in these little Python scripts first, and then go tackle the same concept in C++ with it's much more cumbersome syntax. I may be the only firmware engineer who has written my own functor or used a back_inserter. How cool is that? (no really, I'm not sure...is it cool if nobody else can understand my code? Hmmm...)

In the end, I know, this is probably old news for many of you, but Python rocks. Go spend an hour and automate something tedious with it.

Comments

Post a Comment

Popular posts from this blog

SystemVerilog Fork Disable "Gotchas"

This is a long post with a lot of SystemVerilog code. The purpose of this entry is to hopefully save you from beating your head against the wall trying to figure out some of the subtleties of SystemVerilog processes (basically, threads). Subtleties like these are commonly referred to in the industry as "Gotchas" which makes them sound so playful and fun, but they really aren't either. I encourage you to run these examples with your simulator (if you have access to one) so that a) you can see the results first hand and better internalize what's going on, and b) you can tell me in the comments if this code works fine for you and I'll know I should go complain to my simulator vendor. OK, I'll start with a warm-up that everyone who writes any Verilog or SystemVerilog at all should be aware of, tasks are static by default. If you do this: module top; task do_stuff ( int wait_time); #wait_time $display ( "waited %0d, then did stuff" , wa...
Read more

'git revert' Is Not Equivalent To 'svn revert'

I just learned that if you have some changes in your working tree that you want to get rid of, you don't type 'git revert' like you might guess. No, that's what cvs, subversion, mercurial, and bazaar (to name a few) use revert to mean, but not git. With git, revert is used to undo actual commits. Thankfully, you can undo your revert with another 'git revert', I just learned. So let me repeat to be clear, if you have changes to your working files that you want to abandon, DO NOT do this: git revert HEAD That will undo your last commit. Do this instead: git reset --hard HEAD I'm glad I have that straightened out now. I'm wondering if /etc was really a good place for me to start out playing with git. UPDATE: Nearly two years later and I'm still getting comments on this. I'm glad I've been able to help people out this way. The discussion in the comments is good, and one thing I'd like to point out is that I now always use...
Read more

SystemVerilog Streaming Operator: Knowing Right from Left

SystemVerilog has this cool feature that is very handy for converting one type of collection of bits into another type of collection of bits.  It's the streaming operator.  Or the streaming concatenation operator.  Or maybe it's the concatenation of streaming expressions (it's also called pack/unpack parenthetically).  Whatever you want to call it, it's nice.  If you have an array of bytes and you want to turn it into an int, or an array of ints that you want to turn into an array of bytes, or if you have a class instance that you want to turn into a stream of bits, then streaming is amazing.  What used to require a mess of nested for-loops can now be done with a concise single line of code. As nice as it is, getting the hang of the streaming operator is tough.  The SystemVerilog 1800-2012 LRM isn't totally clear (at least to me) on the details of how they work.  The statement from the LRM that really got me was this, "The stream_operator <...
Read more