Scripting a Simple Download Scheduler

Many of us have an Internet connection that offers unlimited downloads at odd hours, usually around midnight—mine is between 2 a.m. and 6 a.m. What most of us do is wake up late at night to download whatever heavy files we wish to, in order to capitalise on this unlimited bandwidth. However, it becomes quite inconvenient to get up every day in the wee hours and set up things in order to download. Then there’s the additional headache of again getting up when the time slot for unlimited bandwidth ends and cancel the downloads. If only there was a way to do these things automatically without the need to manually start and stop the download processes.

In this article, we’ll talk about how we can write a shell script to create a simple download scheduler for domestic purposes. Additionally, we’ll explore a way in which we could make the PC automatically connect to the Internet, provided we schedule the job at the required time (scheduling will be described later).

Note that by writing a script, we’d in no way write any functions and programs, but combine some features of the following utilities:

• lynx-cur – a command-line Web browser
• wget – the non-interactive network downloader
• Bash shell – for running our script, of course
• at – a utility used for scheduling jobs

Make sure your PC has all the above utilities before going further. wget, at and Bash shell, might be there already. However, the same might not be the case with lynx-cur. If it is not present, run the following command:

sudo apt-get install lynx-cur

How to go about it

The following tasks will be performed by our scheduler—all automatically:

1. Connect to the Internet at the scheduled time
2. Download the file present at the provided URL
3. Log out from the Internet service at the scheduled time
4. Shut down our system at the scheduled time

Before we dive into the subject, there are two things to make a note of: make sure you log in to the system as the root user, and save all scripts and text files that we will create in a single directory.

Also, the first and third tasks listed above—connecting to the Internet and performing the logout—apply only for those connections that demand users to log in by submitting their user name and password into a Web page located at the ISP’s server, and to log out by clicking on a logout link (for example, Sify).

People with always-on connections (like Airtel, BSNL, etc) need not worry about these two tasks as most of these connections ensure connectivity as soon as the modem/router is connected to the system. Those belonging to this category only need to worry about downloading and shutting down the system at the right time.

We will run three independent scripts: start.sh, logout.sh and shutdown.sh—each from three different terminal windows. start.sh will control logging in to the Net and starting the download. logout.sh will log out the system, and shutdown.sh will shut down the system.

A note on lynx-cur

lynx-cur is a command-line browser—there’s no mouse and no graphics here; you can only browse using keystrokes. If we are able to create a file that contains the sequence of key-strokes required to navigate a particular Web page, and if we are able to effectively transfer this data to the browser, then logging in and out are easy.

This is where lynx-cur comes to our aid. It has the striking feature of reading the key strokes from a file and then sequentially applying them on the given Web page, creating the same effect as if we were manually doing it. All you need to do is to provide an option: -cmd_script=<file name> while invoking the browser to browse a page. Lynx will read the key strokes from the specified file. For example:

lynx -cmd_script=<file name> <URL of the web page>

So the problem of an automatic login and logout is resolved provided we have a file that has the key-strokes data logged exactly in the same way the browser would like to read and interpret it.

But how do we create such a file? Lynx has an answer for this one too. Just like the -cmd_script option, Lynx provides another option called -cmd_log. With this option, the browser will log the key strokes we execute while browsing to a specified file (after it gets invoked and till we quit). For example:

lynx -cmd_log=<file name> <URL of web page>