Hack This: Automate Your Operating System with Launchd

Welcome back to Hack This. It's been a few weeks. For that I am sorry. To be honest, I hit the end of my list. Which is good because, hey, now you can consume a bunch more Hack This entries, and bad because coming up with topics isn't always easy. To that end, I can't encourage you enough to submit tutorial suggestions to me at michael.byrne@vice.com and-or @everydayelk. I'm also not just looking for programming-centric topics, but am aiming to expand the scope of Hack This to include more everyday how-tos for navigating technology. That's actually how the thing started waaayyyy back in Motherboard's early days; I just tend to think in code.

This edition of Hack This is going to be kind of in-between. Basically, we're going to be setting up programs or scripts to execute automatically at specific time intervals. Scheduling, automation. At the minimum, it will involve a wee bit of code, but, depending on what you want to accomplish, it could also involve very elaborate shell scripts glueing together even more elaborate scripts (or regular old programs). We'll focus here on a Mac OSX utility called launchd, but will make sure to nod to the Windows and Unix equivalents, which shouldn't be all that different. Prerequisites are mostly nil, but you'd do well to read the post about using the command line here.

There are computer-based tasks you do—or should do, but don't—that are the same every time, or are close enough to being the same. This might mean digital housekeeping, like deleting files or syncing things to the cloud, or it might mean checking a data feed for new updates or verifying the status of some websites that you administer. A lot of these things you might not even think of as being repetitive and automate-able. For example, every couple of weeks I check Boomkat for recommended new releases in a few specific genres and make a list of things to listen to; I do something similar with Maximum Rocknroll top-ten lists. There's no reason I couldn't just write a script that does this and then emails me a list of stuff to listen to every couple of weeks. Given the amount of time that we spend on these machines, not automating almost just seems like a form of procrastination.

First off, launchd is much bigger than what we'll be using it here for. This utility is what your (Mac OSX) system uses to boot—first firmware launches the kernel, which then launches launchd. Launchd then runs through the contents of a couple of directories looking for stuff to run, first launching required background processes and then startup user processes. As things go along, it will stand ready with an army of daemons ready to launch on-demand if they're needed to support some possible user action.

The jobs that launchd needs to complete (the programs or scripts it is to run) are specified in files known as a property lists or plist files. A plist is a key-value list (like a Python dictionary) where the keys are the names of different job properties, such as the file-system path to the executable file to be run; the arguments the executable needs to run properly; the id of the user asking for the program to run; and, if applicable, a future date and time for when the program is to run.

Pretty much anything. Any program you can open with the click of a mouse you can open using the command line (open -a program_name). And anything you can open on the command line, you can launch with launchd. This will be most useful to you with shell scripts, which are just listings of shell commands (command line commands) that are packaged into a single file and do (automate, perhaps) something useful.

Here, let's automate something fairly simple. For me, it would be useful to have a sort of system janitor that comes around to certain file directories and trashes certain sorts of files. For example, in my day to day work for Motherboard, I download loads of image files, and, after a while, I wind up with a downloads folder that's clogged with screengrabs, Shutterstock photos, public domain photos, and so on. I need these photos for like five minutes after I actually download them, but they wind up sitting around and accumulating. Fortunately, we can solve this digital clutter problem with automation.

So, we're going to make a Bash script that does this file-deleting task and then create a corresponding plist file telling launchd how and when to run it.

First, we need the script.

This is just an example of something that might be useful, but I'll quickly explain what it actually does. It's a single line.

find $HOME/Downloads/ -maxdepth 1 -type f -name "*.jpg" -delete

I'm using the shell command find to match every file (specified f for file following to the -type argument) in the given directory (but not any further subdirectories, per the -maxdepth argument) with the .jpg extension (an asterisk, or wildcard, means "anything"). The final -delete argument nukes the results of the find.

It should be immediately apparent that this is a sketchy way to do things. If we just run this every day at a specific time, we're eventually going to fuck up and have something in our Downloads folder that we don't want to delete. I solved this by adding a check, which will make the user approve the deletion first. This seems reasonable.

We need to use Apple's kind of janky built-in scripting language AppleScript for this, unfortunately. This will allow us to show a dialog requiring the user to click a button before deleting the files. AppleScript is a whole other deal, so I'll just give you the code here (which is sourced from Stack Exchange):

set timeoutInSeconds to 60 set abortOnTimeout to true tell application (path to frontmost application as text) try set dialogResult to display dialog "Do you want to clean downloads folder?" default button 2 giving up after timeoutInSeconds on error number -128 return end try end tell if gave up of dialogResult and abortOnTimeout then return end if do shell script "/Users/michaelbyrne/cleanup.sh"

Basically, if the user clicks cancel or doesn't respond within a set amount of time, the dialog will close and the job will not run. We can run the above script (which I called "cleaner.scpt") from the Bash shell with the command osascript cleaner.scpt.

All of these script files, by the way, are just text files and can be edited in and saved from any old text editor. It's ultimately up to us to tell the computer how they should be interpreted. If I have a Bash script with a ".sh" file extension and I'm currently operating within a Bash shell session (just type "Bash" into an OSX Terminal window), I can run it just by entering the filename prefixed with its filesystem path and hitting enter. If I'm in Bash or another shell (read: command line-based program), I'll need to specify that my AppleScript script is to be run with the AppleScript interpreter. That's the "osascript" command above. (AppleScript is a Open Scripting Architecture language and is thus interpreted by an OSA language interpreter, osascript—but really don't worry about it.) Likewise, if I had a Python script, I would need to run it with the "python" command.

By default, you should (on a Mac) have a directory called LaunchAgents in your user Library directory ($HOME/Library/LaunchAgents). Find it. This is where we'll put the plist files launchd needs to run scheduled jobs. You probably already have a couple of plist files in there for installed applications that automatically start when you login to your computer. In mine I already have plists for Adobe, Spotify, and Google.

A plist file is actually an XML file, which is a file containing identifiers enclosed by tags specifying the types/meanings of those identifiers. If you remember, a plist file contains a list of key-value pairs. One such pair might look like this, where "label" is the name I've given the scheduled job (which is a different name than that of the script itself). The label is of type key while the value is of type string.

label

cleaner

Here is the complete plist file for my job. It's scheduled to run every day at 4:00 PM. Don't get freaked out by the XML syntax.

Label cleanme.job ProgramArguments /usr/bin/osascript /Users/michaelbyrne/cleaner.scpt RunAtLoad StartCalendarInterval Hour 16 Minute 00

You can see that the value needed to specify a scheduled job is actually an array of values, which is like a list. In this case, there's just one entry in the list, which is itself another list, this time of the dictionary type. The dictionary contains more key-value pairs, specifying the hour and minute that our job is to run every day.

Note also the array containing the program arguments. Often, when running a shell script we need to provide some additional information, which is usually just listed on the command line after the name of the command. In our XML array, we just list out the items that we would normally be listing on the command line. Here, we need to specify that our script is being run as AppleScript, which is the first argument "osascript."

I have this plist file saved in LaunchAgents as "cleaner.job.plist."

With the file saved in the proper directory, we're still not quite done. We need to load the job, which we can accomplish with the Apple command line utility launchctrl.

launchctl load ~/Library/LaunchAgents/YOURJOB.plist

You can verify that your job is loaded by running the list command:

launchctl list

This will bring up every job that launchd maintains, which is a lot, but you should still be able to find yours. Using launchctl, we can also pull the trigger on our scheduled job at any time by using launchctl start my_job. Discontinuing the job is as easy as running launchctl unload my_job (which leaves the plist file intact and reloadable).

Unsurprisingly, there is software that can handle plist files for us. This is LaunchControl, a small program that offers a GUI for dealing with scheduled launchd jobs, effectively replacing the command line utility launchctl. And, given the small bit of background above on using launchctl, navigating LaunchControl should be pretty intuitive. Rather than deal with ungainly XML files, LaunchControl allows us to drag and drop plist properties, including scheduling, arguments, environment variables, etc.

LaunchControl isn't free. It costs $10, but this is on the honor system (there are no keys or serial numbers or anything). That said, for simple scheduling tasks as in our example, I think launchctl is just fine.

Launchd is supposed to have superseded a more general Unix utility called Cron, which is still available on OSX (and other Unix-based systems). Rather than plist files, it's based on crontab files, which are lists of shell scripts/shell commands (jobs) wherein each job is prefaced by a CRON expression. A CRON expression is simply a string of six single-character fields denoting when and how often a job is to be executed.

Here's what our example would look like using cron:

0 16 * * * osascript /Users/michaelbyrne/cleaner.scpt

So, we're again running the job at 1600 hours (4:00 PM), and leaving the date and day of the week information blank, which means that it will run daily.

While command-line scheduling tools for Windows exist, the canonical method is using a built-in app called Task Scheduler. I don't have a Windows computer so I can't run through the process with you, but it looks to be pretty simple (and akin to using LaunchControl for scheduling). It also seems to let you build the jobs themselves via GUI, bypassing the need to write scripts for common/non-custom tasks.

Mac OSX comes with a utility called Automator, which looks a lot like the aforementioned Task Scheduler. It's GUI-based and allows users to skip the shell scripting and command lining for building and scheduling tasks, though it has capabilities well beyond basic scheduling. I've dorked around with it a bit (for task scheduling) and mostly come away frustrated and missing the open canvas of automation via shell scripting. That said, I can't tell you to avoid it.

In writing this, I've realized I need to do a deeper post on shell scripting itself and what we can do with it. My file-deleting example is simple by design, but things can get much, much cooler. Imagine, for example, what we could do with automation and the Twitter API or a web scraper. Fun stuff.

More Hack This:

Hack This: How to Consult Google's Machine Learning Oracle

Hack This: How to Start an IRC Channel

Hack This: An Overdue Python Primer