Hack This: What To Do When Shit Just Won't Work

Tutorial culture at large has little interest in your fallibility as a learner. That you will almost certainly fuck it up at some point goes unspoken, but at the same time you will find a scarcity of guidance as to what you should do when that happens. There are a bazillion questions, most with several good answers, posted to Stack Overflow by green programmers—often quite lazy green programmers—about how to fix such and such very specific bug or error, but there is much less out there about general approaches to the inevitable brick walls that every coder will eventually run into.

Testing and debugging is a vast wing of software engineering. It also happens to be very underappreciated. Perhaps that is in some part because software testers rarely come bearing good news; the quality assurance engineer's job isn't to hand out gold stars but is instead to tell you that, sorry, dog, it's bruk. Fix it.

Software testing is a surprisingly interesting and theory-heavy field, as it turns out. It can also offer some highly useful insights into debugging even the simplest Python script hacks. And rest assured: those hacks will invariably need debugging. So, with that in mind, here are some pointers on fixing that junk before it drives you to one or more of the following: a) posting a bad question on Stack Overflow that will make people yell at you, b) literally pulling your hair out, or, worst of all, c) giving up.

This can be a powerful allure. You may be eyeball-deep in some program or script before something, somewhere goes off the rails. With every obvious fix exhausted and every usual suspected investigated, the mess of code in front of you all starts to look like the enemy. Maybe then a drastic code-ectomy is in order. Maybe just starting over is easier.

The thing about starting over is that, compared to diving yet again into a tangle of apparently malformed code, it feels really good. It feels clean, fresh. You may be back at the beginning, but at least everything is working again. Ahhhh.

But, by starting over, you've missed an opportunity for learning a thing. And that's a guarantee that you will make the same mistake again in the future.

At the same time, while starting over might feel pretty good initially, you also won't be able to shake that nagging feeling that you bailed. Meanwhile, you will have deprived yourself of one of the most potent thrills in not just programming and software engineering, but science and engineering in general: figuring a thing out.

The reality is that the bug is findable and fixable. It is also not nearly as buried as it might seem.

Which brings us to:

It's easy to get lost within the labyrinth that is your own code. It doesn't feel good: a hotness around the temples, an almost nauseating disorientation. You've for sure reached this state when you start trying fixes that you know won't work.

These are the fixes that are worse than shots in the dark.

In this state of mind, you are subjecting yourself to an unnecessary hell while most likely only adding distance between yourself and whatever the real problem (and solution) is. Do this: stop, turn off the computer, and walk around outside for like 10 minutes. Maybe drink some water. Clear your head.

Now, get back to work. With some perspective regained, the chances are much, much better that you'll figure this out.

To be clear, I'm mostly talking here about the sorts of bugs and errors that aren't immediately pointed out by a compiler or interpreter, which is often the case if the error is syntactical. The real bugs are the ones that quietly mess things up, causing the program to blow up somewhere down the line ("downstream"). These are the bugs that require hunting and tracking. They are syntactically correct, but are programmatically incorrect. The effect of a program bug can be very far removed from its cause.

This is why it can feel so good just to gut the thing and start over. Fortunately, there's something of a compromise debugging technique. It's crude and not always effective, but often I'm able to find a bug by progressively rewinding my own code. Basically, I work backward, muting sections of code one by one using comments. Eventually, I'll wind up commenting out the source of the bug, which will become apparent when the program starts working again.

Of course, the program will only be "working" in its earlier form, but now you can step forward through all of the code sections you'd silenced before. The code also has to be sufficiently modularized such that it can be rewound into earlier functional components in the first place and still run. (Imagine a staircase of program versions, with each step adding some new functionality.) This is not a guarantee.

This is the OG debugging technique and was implemented formally in early versions of BASIC as the TRON command (for TRace ON). The TRON command itself is obsolete, but the idea behind it remains probably the most popular debugging technique used by programmers and engineers. Which makes sense because it's incredibly simple and effective.

The basic idea is to write statements into your code that regularly output what the program is doing and what its current state is. Print statements vary a bit language to language, but it's the same concept all around. An example in pseudocode would look something like this:

print "calling method XplusY() with x = aNumber and y = anotherNumber"; result = XplusY(aNumber,anotherNumber); print "the function returned result";

So, we're just printing out what values we're giving the function (which is supposed to add together two values and return the resulting value) and then we print what result it gave us. Without the print statements, the function might otherwise just run and hand off its result to some other variable which might have a bunch of other stuff happen to it as part of the program's execution before any feedback actually made it to the screen, if it ever did at all. Here, we can just look back at our output to see if XplusY is indeed behaving as it should. You could do this line by line through your entire program if you really wanted to. I've come awfully close.

At least not right away.

As you get into this, you will be amazed at the volume of forum posts that exist for each and every slight problem that your code might throw at you. Every question has been answered and most have them been answered several times. Most of this answering occurs at Stack Overflow, where there is an army of annoyed-sounding experts ready to reanswer every and any question you might ever have. As you code more and more you will start to notice a particular category of question, which is the question that the questioner has not even bothered to try and answer themselves. Instead, they have pasted whatever bad output their Fibonacci sequence generator (or whatever) spit back at them as it exited with with a return value of 1.

If you're still kind of new to this, the meaning of the 1 return value might not be apparent. This winds up being a nice illustration of what I'm talking about. To get to the bottom of this whole return value business, you should probably start by just searching Wikipedia, which will generally have your back when it comes to comp sci/software engineering/programming topics. You'll wind up at the page for exit status, which is mostly the same thing as a return code. Here, you will actually learn something.

You will not learn something by opening a new Stack Overflow question with this as the content:

You will get downvoted and people will post links at you, which you should have been finding yourself.

My larger point with this one is to do the hard thing and think it through, at least at first. Don't just Google a quick answer. Those quick answers are out there in abundance, but take the time to read some documentation and watch some videos. You'll wind up creating less bugs further down the line.

In 2001, software engineer David Agan published Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems. It's not a technical book by any stretch, but it's become part of the computer science/software engineering canon. I was introduced to it in a university class on software testing.

Agan's thesis is more or less that we can identify software failures by applying the scientific method in intuitive ways. Observe a failure, come up with a hypothesis about that failure, test the hypothesis. Repeat as necessary. The rules are meant to guide you through the process.

Some are more useful for full-on engineering, but for our purposes, a couple of Agan's rules are worth keeping close at hand.

Here is what Agan writes about "seeing the failure":

A curious thing about software bugs is that it winds up being pretty easy to "fix" something by breaking something else. That is, if there is some distance between the actual bug and the observed failure, a programmer might find some way of tweaking the program in between—applying a patch that doesn't patch the bug so much as it does loosely quarantine the bug. Make no mistake, the bug will return.

While you're tying yourself a noose because al;skdja;lskdjf;aklsjdff;falfuckfuckfuck, the problem may well be something incredibly stupid. It might not be a deeply embedded bug at all, but something external that is very obvious but not to you because you're trying to find a bug inside the program that looks like other software bugs. Were you sure to give it the right commands before running it? Did you forget to include a header file?

From Agan:

You can find the book if you want more.

Debugging is its own theoretical universe, but this is just a start. If your email-sending or image-editing Python scripts aren't working, don't despair. Take some deep breaths of outside air (specifically outside air) and know that this is a problem that you are capable of solving. Sometimes that's all it takes.