Lightweight and simple to use, Watir is a browser driver that is great for automating modern browsers. It is built on Ruby but it also works well with other testing frameworks supported by Ruby. In this article, the author describes the installation of Watir and its use in retrieving the results of the CBSE Board exams.
If necessity is the mother of invention, then boredom is the grandmother of automation. Automation refers to the process of performing a task without any attention or input required from the person performing that task. I am referring to tasks that are quite trivial, while at the same time boring, like getting results from a website based on a query or a user selection. Most of the time, things like this can easily be done by Web scraping or vendor APIs, but we can also instruct browsers to do these systematically. In this article, we will use Ruby as the commanding force and Watir-webdriver (Watir stands for Web automation testing in Ruby) as the bridge between Ruby and the browser. Although Watir is mainly used for Web application testing, we can also use it to create useful scripts. And heres a small note: after you finish this article, you may want to get deeper into Watir, and while exploring the Web, you will usually see both Watir and Watir-webdriver; dont get confused. The difference lies in the browser that is driven. Watir only works/drives with Internet Explorer on Windows, while Watir-webdriver drives Firefox and Chrome on Windows, Mac and Linux.
Installation
Lets get started. Since this set-up requires the use of Ruby, the first obvious step is to check for Ruby. If Ruby is installed on your system then your development environment is already prepared (which is no easy task); otherwise, install Ruby. Depending on your operating system and distribution, the procedures steps may differ. When in doubt, refer to the installation documentation available at https://www.ruby-lang.org/en/documentation/installation/. But whichever way you choose, always remember to use version management tools like rbenv, rvm or pik. For Linux and Mac, the best way to install Ruby is through rbenv or rvm and for Windows, it is pik (no longer maintained) or Uru (though, I would advise you to stick with UNIX systems, as managing Ruby in Windows is difficult compared to Mac and Linux). Check the References section for links of detailed tutorials. Since we have prepared our development environment, we can proceed to install libraries for Watir or Gems as we call them in the Ruby world. Open the terminal or the equivalent to install Gems.
To install Watir-webdriver, use the following command:
gem install watir-webdriver
If the following commands throw errors due to lack of privileges (usually encountered on Ubuntu), prefix the command with sudo:
sudo gem install watir-webdriver # Use this if previous command doesnt work
To install Watir, issue the following command:
gem install watir # Prefix with sudo if necessary
Fire up irb or pry (whichever suits you) to load the Watir-webdriver gem:
require watir-webdriver
Now, to invoke a new browser, use the following command:
browser = Watir::Browser.new :firefox # Default is firefox
Now, lets do it for Chrome, as follows:
browser = Watir::Browser.new :chrome
You may get an error message similar to whats shown in Figure 2.
The error message says that it cant find the chromedriver executable, which is required to drive Chrome. This can be fixed by downloading the latest Chrome driver from the http://chromedriver.storage.googleapis.com/index.html and adding the executable to our PATH (which specifies the list of directories to be searched to find the program). Download the latest chromedriver, which at the time of writing this article was 2.13. The latest version can be checked by visiting the link: http://chromedriver.storage.googleapis.com/LATEST_RELEASE. Make sure to download the correct executable for your system. Since I am running the 64-bit version of Chrome on Linux, I selected chromedriver_linux64.zip. Download the latest version (2.13) and extract it into a folder named driver. The following commands will do the trick:
whet http://chromedriver.storage.googleapis.com/2.13/chromedriver_linux64.zip unzip chromedriver_linux64.zip -d driver
Now, we need to add a PATH for the Chrome driver, as follows:
echo export PATH=$PATH:$HOME/driver >>. .bashrc
Since we now have browser support, we can try to do a few things. From now on, we will call our browser slacko (our virtual pet). Save this script as test_1.rb or whatever name you like.
require watir-webdriver #Include the gem slacko = Watir::Browser.new :chrome #You can also use firefox slacko.goto google.co.in #Loads the google.com slacko.text_field(:name => q).set open source #Sets the query field (having class name q) to open source slacko.button(:name => btnG).click puts slack.title #Prints the title of the page slacko.screenshot.save open.png #Saves the screenshot of the page to open.png
Run the script with the following command:
ruby test_1.rb # substitute with the name of the script
Finding class attributes like names or an ID can be done using developer tools, since almost all major browsers come with one. In Chrome, doing this is as simple as clicking the Inspect element from the right-click menu.
Lets apply this to do something useful.
Result retriever
Watir can be used to automate the retrieval of results for an examination from a large pool of numbers (usually roll numbers). CBSE is one of the boards for education in India and the result day breaks many hearts. Lets write a simple script to retrieve the exams results list for a given range of numbers.
Disclaimer: The following scripts are presented for educational purposes only. While using them, use the resultant data with utmost care and responsibility, and respect the privacy of others. Neither the author nor OSFY are responsible for any misuse.
For the sake of simplicity, lets target the 2013 results of CBSE Class XII.
Heres the complete script:
require watir-webdriver
range = (1..90) # Replace numbers as per use
slacko = Watir::Browser.new :firefox #Makes a firefox pet
slacko.goto http://resultsarchives.nic.in/cbseresults/cbseresults2013/class12/cbse122013.htm #Loads the target page
range.each do |number|
slacko.text_field(:name => regno).set number #Sets the input with the current number in range
slacko.button(:name => B1).click #Clicks the submit button
slacko.screenshot.save #{number}.png #Saves the screenshot of result as number.png
slacko.back #Loads the previous page
end
If there are no errors and the script runs fine, you will see a list of screenshots of results arranged by roll numbers. Since you will have a lot of data, use it carefully and dont use it to harm anybody. This could be particularly useful for schools that want to digitally archive the results of their students.
Going headless
The results retrieval script is good but browsers use resources. We can replace Firefox or Chrome with a headless browser (one with no graphical interface) like phantomjs so as to save memory. Headless browsers like phantomjs can accomplish the same task with less resource consumption and also support screenshots.
Installing phantomjs is as easy as downloading the binary and placing it in the path.
The following commands will come handy:
#For 64 bit Linux systems wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-1.9.8-linux-x86_64.tar.bz2tar xvjf phantomjs-1.9.8-linux-x86_64 cd phantomjs-1.9.8-linux-x86_64/bin/ sudo ln -sf phantomjs /usr/local/share/phantomjs sudo ln -sf phantomjs /usr/local/bin/phantomjs sudo ln -sf phantomjs /usr/bin/phantomjs
If all goes right, a new phantomjs instance can be created by the following code:
require watir-webdriver slacko = Watir::Browser.new :phantomjs #Do crazy things with slacko :)
References
[1] Ruby Installation Documentation: https://www.ruby-lang.org/en/documentation/installation/
[2] Ruby Installer and Pik for Windows (Pik is no longer maintained, so proceed with care): http://misheska.com/blog/2013/03/10/using-pik-to-manage-multiple-versions-of-ruby-on-windows/
[3] Uru Wiki: https://bitbucket.org/jonforums/uru/wiki/Home
[4] Rbenv on Mac and Linux: http://misheska.com/blog/2013/06/15/using-rbenv-to-manage-multiple-versions-of-ruby/
[5] Link to download the Chrome driver: http://chromedriver.storage.googleapis.com/index.html
[6] 1 Watir cheat sheets and references: https://awetest.zendesk.com/hc/en-us/articles/201883796-Watir-Webdriver-Cheatsheet
[6]. 2 https://github.com/watir/watir/wiki/Cheat-Sheet
[7] Phantomjs installation instructions: https://gist.github.com/julionc/7476620
[8] CBSE link used for demonstration: http://resultsarchives.nic.in/cbseresults/cbseresults2013/class12/cbse122013.htm
[9] Phantomjs: http://phantomjs.org/download.html
