FYI.

This story is over 5 years old.

Tech

Know Your Language: The Ghost in the Shell Script

Shell scripts are the programs that tell other programs to do stuff, but that's just the start.

What's the difference between programming and scripting anyway?

There are a billion variations of this question posted across the internet and the answer is not always as simple as it might seem. The popularity of Python, a language which is unusually adept at both programming and scripting, and JavaScript (for analogous reasons) certainty have a lot to do with the muddying, but the distinction persists because, well, there is a distinction.

Advertisement

The programming vs. scripting difference is easy to see when we consider shell scripting. Classically, a shell script is a program (a collection of instructions or commands) intended to run within a Unix shell environment (by "Unix" assume that I mean actual Unix along with its many open-source analogs, e.g. Linux). There are a lot of shells actually and they can be based on a lot of different languages—Python, Java, C, etc.—but the general idea of a command line interpreter is more or less the same.

The Shell

If you haven't used a command line very much, it's worth taking a small amount of time to just poke around, whatever your operating system. Windows and OSX both have command line interfaces of varying capabilities. In Windows you can find it by searching "cmd" and in OSX it's called Terminal. (The Windows shell, which is not Unix-based but mimics it, is a lot less useful, for a variety of reasons.)

The idea is that these are just non-graphical portals into your computer. A command line is a hyper-minimal version of an operating system, but with all of the same capabilities. This is what an operating system used to look like exclusively. The ancestor to all modern Unix systems came around 1970 and the introduction of Digital Equipment Corporation's PDP-11, considered to be the first microcomputer.

Unix then looked like Unix now, almost 50 years later. Unix in 2015 even still comes with a "rewind" command, which is how a user might tell a computer to rewind its cassette tape memory.

Advertisement

Within a shell (any shell for any operating system), programs are launched by, well, typing the name of the program. And shells also have a bunch of small bits of programs/utilities built in, like, "ls" ("dir" in Windows), which will list all of the files in a particular folder (directory). Or maybe you've had to reset your IP address using "ipconfig" or "ifconfig." With "cat" I can create or display a small file and with "grep" I can search a file for a regular expression. These are all just computer programs/utilities, but they can only be run from the command line, which is all they require.

Working with shells is fun because it's fast and immediate, for one thing. Rather than working with a piece of software, you're working with one aspect of the giant hydra that is an operating system. There's no waiting around for an independent program to load, nor does a shell command have to "ask" the operating system for resources. It is the operating system!

Hello, World

A shell script is a way of packaging a bunch of commands together, exploiting the immediacy and speed of the operating system and the terseness of shell commands to do more complicated tasks.

Here is a "Hello, World" script. It's meant to be interpreted using the the super-popular Bash shell, but it could be run just as easily with another Unix or Unix-like shell.

!/bin/bash # My first script echo "Hello World!"

This looks an awful lot like a computer program and, in a sense, it is. A critical difference, and what is usually used to demarcate between scripts and programs, is compilation. When I write some code in C++ or another typical programming language, this is the step in which a piece of software takes all of my code and converts it into machine-readable instructions. The result will be an executable file—a piece of software. To run the software, the operating system will load it all into memory and weave it into a queue of executing processes. These processes will all share the computer's processor(s).

Advertisement

A script, however, doesn't have this step. When I write a shell script, I'm taking a bunch of pieces that are already just there as functions of the operating system or are other scripts. They're already compiled and, in a sense, you can think of them as already running. All the scripter is doing is taking these moving gears and lashing them together in useful ways. Scripting languages are often referred to as "glue" languages, because they take existing programs and stick them together. A script doesn't "run" so much as tell other things to run at such and such time in such and such order.

The script also serves to "pipeline" data from unit to unit. If I run an ls command, for example, which will output a list of files within the current directory, I can use a single character ("|") to direct that output to the input of another unit, which might write the names of those files into a text file.

A shell script could theoretically be as elaborate as any program, but that sort of defeats the purpose.

Scripting vs. shell scripting

Not all scripts are shell scripts. There are entire independent scripting languages, such as Python and Lua, and they function in a similar way. The difference is that rather than using the operating system as an execution environment, a scripting language will have its own. Python code can either be packaged into complete programs (not so much scripts) or it can be used as a scripting language, in which small programs dictate the behavior of other programs. Because it's just taking other complete programs and stringing them together, a Python script doesn't have to be compiled. It's just glue.

Advertisement

Shell scripts are meant to be super-lightweight things, but a Python script can become very elaborate and the capabilities of the Python interpreter are relatively vast and ever-expanding.

More scripting definitions

I thought maybe it'd be helpful to offer up some definitions from around the internet because scripting really is one of those endlessly debated and pontificated-on things.

Software engineer Nael Shawwa:

So what is Scripting? I think it is when you use a tool or combination of tools to automate a task or series of tasks—regardless of the language it is written in. I can be lazy, I would script cooking breakfast if I could.

Phoenix 750:

scripting languages (like Bash, Python, Ruby…) are mainly designed to automate certain tasks (especially Bash). a lot of hackers use Python to write exploits (though i prefer doing it in Metasploit). scripting languages like python can have an extensive amount of functionality, but scripting languages have one big downside: they are slow. that is because unlike programming languages, scripts are interpreted, meaning that the program executes command per command, rather than throwing all commands together and translating it to machine code.

Herman Venter, MSDN Blogs:

Programs are sets of instructions carried out directly by computer hardware, perhaps with the cooperation of a runtime and/or operating system. Scripts, on the other hand, are sets of instructions carried out by other programs, such as text editors, command line processors and most famously, Web Browsers. At a high enough level of abstraction, there is no conceptual difference between "scripting an operating system" and "scripting a text editor", but in practice the details matter and they are different enough to have a material affect the design of a historical scripting languages.

Getting started

Scripting is a great way to get started with programming, generally. With something like Python it's easy to do very cool and hackery things without much background. If you wanted to hack your Twitter, for example, there are loads of clever scripts for doing all kinds of stuff. Other obvious Python utilities might include web scraping, data syncing, command line Googling, HTML verification and maintenance, collecting data for/from any number of pipelines/APIs, live-coding beats, and on and on and on. Thanks to Python, you could be flooding Twitter and beyond with bots by dinnertime. Download Python 3 here.

For shell scripting, you can screw around with your Windows command line, but with something like Cygwin, you can run a Linux-like environment on the same machine and have the same Linux-like capabilities. On a Mac, you're already running Unix and so shell scripting is more natural.

Read more Know Your Language.