The power of scripting lies in the fact that you get to program with commands you already know, from various computer languages. Scripting provides a means to substitute really complicated and convoluted commands and if there is something repetitive, then a script can probably remove the tedium in your work.
Imagine that you are doing some important task on your computer system and, suddenly, you get a black screen with something being automatically typed on it. I am sure many of us have experienced such system reboots that take place on their own due to some hardware or software issues. When this happens, our pulse rate shoots up, and we hope and pray that the task that was being performed has been saved, or else, everything will have to be started afresh.
Have we ever given a thought to how all this happens on its own? Well, it’s shell scripting that drives such processes automatically. A newbie in the computer programming arena generally gets anxious just seeing the black screen on the computer, assuming the programming involved is quite difficult.
In shell scripts, the term ‘shell’ defines the interface between the user and the different services of an operating system such as Linux, UNIX, Windows (to some extent) or Mac. Apart from the shell, the kernel is the other main component of an operating system. It’s the kernel that makes the communication between the hardware and software possible. The interface can be created by a shell through different command line interfaces (CLI) or in a graphical manner (using the GUI). A shell takes input from us in the form of commands, processes it and then gives an output. It can be accessed by a terminal, which runs it.
Whenever we run the terminal, the shell actually issues a command prompt, where we can type our own input, which is then executed when we hit the Enter key. The output is thereafter displayed on the terminal. The shell basically wraps around the delicate interior of an operating system, protecting it from any accidental damage and, hence, the name. There are different types of a shell, such as Korn shell (ksh), Bourne shell (sh), C shell (csh), Bourne Again shell (bash), a remote shell (rsh), etc. A script written in any such environment is referred to as a shell script.
The different operations that can be performed by shell scripts include file manipulation, execution of programs, and printing text. A script that sets up the specific environment, runs the program, does clean-ups if necessary, does logging, etc, is basically called a wrapper, which refers to the automated mode of running a specific operating system shell. In different operating systems, shell scripts are referred to by different names such as batch files (OS/2, MSDos-Win95 stream), command procedures (VMS), and shell scripts (Windows NT stream and third-party derivatives). Mainframe operating systems are associated with a different set of terms.
Shell scripts help us to program different commands in chains and have the system execute them as a scripted event —the same as batch files. They also allow us to perform far more useful functions, like command substitution. We can invoke a command, like date, and then use its output as part of a file-naming scheme. We can even automate backups, and each copied file can have the value of the current date appended to the end of its name. Shell scripts are not just invocations of different commands either, but are programs in their own right. Shell scripting also allows us to use different programming functions – such as if/ then/ else statements, ‘for’ loops, and so on – directly within our operating system’s interface. We don’t have to learn another language because we are using what we already know—the command line.
Different features of shell scripts
Let’s have a look at the different features of shell scripts.
Running batch jobs: Shell scripts allow different sets of commands that are entered manually at a command line interface to be executed automatically, without having to wait for a user to separately trigger each stage of the sequence. For example, in a directory that has three C source code files, instead of manually running the four commands which are required to build the final program from them, one could create a C shell script, and keep it in the directory along with the C source code files, which would ultimately compile them automatically. The script would further allow a user to save the file being edited, pause the editor, and then just run the shell script to create the updated program, test that, and return to the editor again.
Different shortcuts to perform specific functions: We can use the different available shortcuts in shell scripting to perform some specific action. These shortcuts are short commands written for specific utilities. They are like inbuilt functions, which we use in different programming languages. A shell script can also provide convenient variations of a system command, where different special environment settings, post-processing or command options apply automatically, but in such a way that they allow the new script to still act as a complete normal UNIX command. For example, to create a version of the ls command to list different files, it can be given a shorter command name of ‘l’. This shorter command will be normally saved in the bin directory of a user as /home/username/bin/l and a default set of different command options will be pre-supplied. The user can then use ‘l’ for most commonly used short listings.
Generalisation: Simple batch jobs are usual for isolated tasks, but at the same time, using shell loops, variables and tests provides much more flexibility to users. Let’s look at a bash file (shell script) to convert different JPEG images to PNG images, in which the names of images are provided on the command line—possibly using wildcards. Instead of each of the names being listed within the script, they can be created with this file, typically saved in /home/username/bin/jpg2png. So we can directly run this command on an entire directory of JPEG images with just /home/username/bin/jpg2png *.jpg.
Flavour of programming: Many modern shell scripts also supply various features which are usually found only in more sophisticated programming languages, such as control-flow constructs, comments, variables, subroutines, arrays and so on. With such features available, it is quite possible to write reasonably sophisticated applications using shell scripts. However, they are still limited by the fact that most shell scripting languages have little or even no support for data typing systems, threading, classes, complex math and other such common language features. They are also much slower than the compiled code or interpreted languages that are written with speed as a performance goal. The standard UNIX tools, Awk and Sed, provide extra capabilities for shell programming. Perl can also be embedded in different shell scripts as can other scripting languages such as Tcl, etc.
Verisimilitude: A key trait of different shell scripts is that the invocation of their interpreter is considered as a core operating system feature. So, instead of a user’s shell only being able to run different scripts in that shell’s language or a shell script only having directives of its interpreter handled correctly, if it is run from a shell (both were limitations in the early Bourne shell’s script handling), shell scripts are set up and executed by the OS itself. Nowadays, a modern shell script is not just on the same footing as the system commands, but rather, many of the system commands are actually shell scripts. This extends to the returning exit codes like other system utilities in order to indicate success or failure, and also allows them to be called components of the larger programs, regardless of how all those larger tools are implemented. Like different standard system commands, shell scripts omit any kind of file name extension unless it is intended to be read into a running shell using a special mechanism for this purpose (such as csh’s source or sh’s ‘.’).
Shell scripting on various operating systems
It is possible to leverage the various advantages of shell scripting on different operating systems (other than UNIX and Windows) as well with the help of interoperability software. Let’s have a look at some of these options.
1. Interoperability software like Cygwin, Interix (which is available in the Microsoft Windows Services for UNIX), MKS Toolkit, UWIN (AT&T UNIX for Windows), Hamilton C shell, and others, allow UNIX shell programs to be run on different Windows NT machines and all their successors, with some loss of functionality on the MS-DOS-Windows 95 branch, as well as earlier MKS Toolkit versions for OS/2. At least three DCL implementations for Windows OS – apart from XLNT, a multiple-use scripting language package (which can be used with the command shell), CGI programming and Windows Script Host — are available for these types of systems as well.
2. In addition to all these tools, some OS/2 and Posix functionality can also be used with the corresponding environmental sub-systems of the Windows NT OS series up to Windows 2000 as well. A third 16-bit sub-system, often called the MS-DOS sub-system, uses Command.com which is provided with these OSs to run all the just-mentioned MS-DOS batch files.
3. Different console alternatives such as 4DOS, 4NT, 4OS2, Peter Norton’s NDOS, FreeDOS, and the GUI Take Command, which add the functionality to Windows NT-style Cmd.exe, OS/2’s Cmd.exe, MS-DOS/Windows 95 batch files (run by Command.com), and 4NT, respectively, are similar to the shells that they actually enhance and are much more integrated with the Windows script host. The Windows Script Host comes with three engines (VBScript, VBA and JScript ) that are pre-installed, and to which a number of third-party engines can be added with Perl, Rexx, Python, Tcl and Ruby with pre-defined functions in 4NT. PC DOS is also quite similar to MS-DOS, while DR DOS is quite different. Various earlier versions of Windows NT are able to successfully run contemporary versions of 4OS2 with the help of the OS/2 sub-system.
4. Scripting languages can be extended; for example, Windows NT and MS-DOS/Windows 95/98 type systems allowed for batch/shell programs to call different tools like KixTart, QBasic, Rexx; various Basic, Python and Perl implementations; and the Windows Script Host and all its installed engines. On UNIX and other Posix-compliant systems, Sed and Awk are used to extend the string and the numeric processing ability of shell scripts. Tcl, Rexx, Perl and Python have graphics toolkits that can be used to code different procedures and functions for shell scripts. This poses a speed bottleneck (Fortran, C, and Assembly language are much faster still) and adds functionality such as sockets and other connectivity functions, which are not available in the shell language. VBA and VBScript can be easily used to communicate with databases, spreadsheets, scriptable program of all types, development tools, telecommunications software, graphics tools and other software that can be accessed using the Component Object Model.
Writing your first shell script
Well, we are now at the most interesting part, which is, programming using the shell script. So we will actually follow three simple steps in order to successfully write our first shell script, which is the famous program to display ‘Hello World’. The pre-requisites are:
- You need to have Linux installed on your system.
- You need to have a text editor (select from Vim, kwrite, etc), as well, on your system.
Now, let’s follow the three simple steps.
1. Write the script
A shell script is basically a file that contains ASCII text. In order to create a shell script, we use a text editor, which is nothing but a program, similar to a word processor, which reads and writes different ASCII text files. There are many text editors available for our Linux system, both for the GUI environment and the command line environment. Here is a list of a few:
1. Vi or Vim
2. Emacs
3. Nano
4. Gedit
5. kwrite
Now, we can type in the first script using any of the above text editors, as follows:
#!/bin/bash # My first script echo "Hello World!"
The above script has three lines.
- The first line is a special clue given to the shell in order to indicate what program is used to interpret this script. In this case, its /bin/bash. Other scripting languages such as Awk, Perl, Tcl, Tk, and Python can also use this type of mechanism.
- The second line is just a comment. Everything that comes after a ‘#’ symbol is all ignored by bash. As our scripts become larger and more complicated, comments become important. They are basically used by programmers to explain what’s going on, so that others can understand.
- The last line is the echo command, which just prints whatever is given on the display. Once we have written the above script, we can save this file using any name. Here, we save it as First_Shell_Script.
2. Give the shell permission to execute it
The next thing we need to do is to give the shell permission to execute our script. This is done using the chmod command, as follows:
[me@linuxbox me]$ chmod 755 my_script
The ‘755’ will give us permission to read, write and to execute. Everybody else will just have read and execute permission. If we want our script to be private (so that only we can read and execute), we use ‘700’ instead.
3. Put it at the location where the shell can find it
We now need to run our script by typing the following command:
[me@linuxbox me]$ ./First_Shell_Script
Now, you should see ‘Hello World!’ displayed. If we don’t, then we need to check what directory we saved our script in, so that we can go there and try again.
Before we go further, let’s discuss paths a little bit. When we type in the name of a command, our system does not search the entire computer in order to find where the program is located. That would really take a long time. You will have noticed that we don’t usually have to specify the complete path name to the program we want to run; the shell just seems to know that. The shell maintains a list of different directories where executable files are kept, and it just searches for the directories in that list. If it’s not able to find the program after searching each of the directories in the list, it will throw up the error: Command not found.
This list of directories is called the path. Let us look at some of the commands related to this along with their possible output.
- The command for viewing the list of directories is as follows:
[me@linuxbox me]$ echo $PATH
The output: A colon-separated list of directories which will be searched if a specific path is not given when a command is attempted.
- The command for adding directories to the path, where directory_name is the name of directory we want to add, is:
[me@linuxbox me]$ export PATH=$PATH:directory_name
The output: directory_name will be added to the path.
- The command for creating a bin directory, which is a sub-directory of our home directory, is as follows:
[me@linuxbox me]$ mkdir bin
The output: The bin directory will be created.
Shell scripting can be used in the following cases:
1. To create our own command or tool.
2. To monitor several tasks like disk usage, etc.
3. To take data backup and snapshots.
4. For automatic system booting.
5. For automatic installation of different application packages.
6. To automate different aspects of computer maintenance and user account creation.
7. To create startup scripts for different unattended applications.
8. To kill or start multiple applications together.
