Open source technology enabled Just Dial, India’s leading local search engine, to grow at a much faster rate and achieve a lot more than it planned for.
Every business that signs on to adopt technology hopes it will be a fruitful partnership. Just Dial, India’s leading local search engine, began its relationship with open source technology about three and a half years ago, and has never looked back. It’s grown by leaps and bounds, and now caters to over 25 million unique users across 240 cities in India. It employs over 4,000 people and has more than 125,000 SME advertisers today. And behind this successful business is none other than the “open” hand!
“Open source technology enabled us not only to dream big but live the big dream. It allowed us to scale up in a seamless manner, helping us grow without having to worry much about the procuring part of open source. Second, because of the open architecture, we were able to introduce best practices from various fields and add value to our search process. We used open source tools from sociology, politics, economics, etc, modifying them as per our requirements to get the most out of these tools. In a nutshell, this approach allowed us to extricate more information and give refined search results to our end users,” says Sandipan Chattopadhyay, CTO, Just Dial.
The first spark
In 2007, Just Dial went in for a Web-based server system based on an open architecture. The ambitious project to make a client-facing website provide more than the telephonic service it traditionally offered its customers, triggered Just Dial’s move towards open source technology. The firm had decided to launch two websites — one client-facing and another for internal use.
Back then, Just Dial had a DOS-based system built in FoxPro. The company operated out of eight regional offices that were faced with everyday challenges like taking care of the linguistic aspects of the search process. A centralised call centre approach for the phone service, too, would have resulted in too many linguistic problems. Besides, the company believed in the human touch, wherein technologies ought to work around people and not the other way round (where technology is forced down people).
Thus, an open and distributed architecture was a perfect fit for Just Dial.
“We typically didn’t want our Web service and call centre to be dependant on a client server architecture, as maintaining it would be cumbersome compared to the Web-based open architecture. Our traditional client servers were also having performance issues, slowing down the system. This was hindering prompt service delivery. As we planned our movement to the Web world, we expected as many queries on the website as via the telephone route, which would result in a huge performance pressure on our search algorithms. We knew the traditional client server architecture would collapse and thus opted for open and distributed architecture,” says Chattopadhyay. “Also, the way we leveraged Web 2.0 technology, it was very light on the network and could be easily sustained even on the meagre 10 MB architecture we had back then.”
Alongside this, on the software side, the firm adopted the CLAMP stack, which was a combination of C and the LAMP (Linux, Apache, MySQL, PHP) stack. About 200 servers, connecting 3,500 workstations at Just Dial, are powered by this CLAMP stack today.
“The LAMP stack is well known for its robustness, reliability, etc, which is why we decided to adopt it. We didn’t want to upset or play around with a stack that works so beautifully. For instance, for every kernel module that we wanted to patch onto a Web server, Apache Web support was there. So was the case for MySQL, which fitted perfectly with our search, which is parameterised, and based on advanced artificial intelligence principles. Further, MySQL and Red Hat made an awesome combo. Thus, LAMP was the de facto standard for us. However, we did add a lot of C components into the kernel itself to make the stack even more optimal,” says Chattopadhyay.
Just Dial chose Red Hat Enterprise Linux (RHEL) to power its servers, Ubuntu on the desktop front, MySQL, PostgreSQL and MongoDB for database management, and WebHuddle for Web conferencing and video conferencing. While performance, security of the platform and trust in Red Hat made Just Dial bet on RHEL, the optimal network stack in Ubuntu won Just Dial’s confidence in the Ubuntu OS.
“When we were choosing an open source application platform, we were looking at two criteria — first we wanted assurance about the stability of the vendor and we wanted a secure and high performance platform. With years of experience in open source technology, watching players come, go, merge and demerge, I knew Red Hat was here to stay. The Red Hat stack was also very malleable, allowing us to go to any point, script it off and yet have the stack up and running perfectly well. Lastly, the scalability, and system performance of RHEL was undisputable,” says Chattopadhyay.
Just Dial adopted Ubuntu with a twist for its 3,500 desktops. “We used the Ubuntu OS, but removed parts that a particular set of users didn’t need. For example, we have three to four departments, with each department having a dedicated task. Our CROs (Customer Relations Officers) need the Web browser to be effective and need an optimal network stack. This was done by giving the maximum amount of available RAM so that they could use the cache to the fullest. Alongside, we made the cache for each system specific to the user, and Ubuntu OS allowed us to do that,” says Chattopadhyay.
The company also uses WebHuddle, a conference tool. The cross-platform functionalities, simplicity and easy usage of WebHuddle caught the eye of those at Just Dial. “We were entering new markets and with people getting more tech savvy, we needed a conferencing tool. We wanted an open source one as I believe in freedom, irrespective of whether we do anything with it or not. If I get an idea, I shouldn’t be twiddling my thumbs wondering how to take it forward. WebHuddle provided me that freedom and flexibility. It was also very simple and easy to use, which made it our choice over other open source conferencing tools,” says Chattopadhyay.
Open source — the loyal partner
But did open source turn out to be a staunch supporter to Just Dial?
“Open source has never let Just Dial down. Improved performance, flexibility, cost efficiencies, scalability, and perfect alignment with the firm’s ambitious growth aspirations made this a perfect partnership for Just Dial,” claims Chattopadhyay.
Greater scalability
The open source IT architecture helped Just Dial improve scalability. Since the time the firm started using the RHEL platform and CLAMP stack, it has been able to scale operations by three times. “Three years back, our executives used to attend close to 80,000 calls per day. That figure has crossed 250,000, as of today. We have also been able to scale our IT infrastructure without any hiccups. We have grown from five servers to more than 200, and expect our server count to reach 350 by the year end,” says Chattopadhyay.
A fine performance
Open source technology helped Just Dial extract tremendous performance from search systems and applications. “The platform allowed us to try things that we had never attempted before, helping us refine our search logic. On the old system, we were hitting dead ends related to the network and scalability. Open source freed us from vendor lock-in, and enabled the organisation to choose hardware platforms that delivered flexibility and high performance,” says Chattopadhyay.
The open source architecture seamlessly ran mission critical applications like search logic, and also helped Just Dial improve the performance of its search logic by adding C components. For example, the company used Sphinx (which is a full-text search engine written in code C that’s designed to integrate well with SQL databases) on its LAMP architecture. “When the volume of data increases tremendously, most databases slow down. That is where non-database queries and lexis geographic searches gain relevance. Thus, we adopted Sphinx, a standalone search engine, and its easy integration in LAMP stack gave our search logic a boost. With its built-in data source drivers that support the fetching of data either via direct connection to MySQL, or PostgreSQL, Sphinx helped us achieve fast, size-efficient and relevant full-text search functions,” says Chattopadhyay.
The cost advantage
The open architecture also gave Just Dial a huge cost advantage. “Open source lowered the IT cost burden for us. At that point, if we had chosen Microsoft, we would have bled paying the licensing fees for 4,000 desktops and 200 servers. Today, we only pay Red Hat’s subscription fee, a gold support fee to MySQL, which is chicken feed compared to the licence costs associated with any proprietary vendor. Annually, we have saved at least Rs 80-90 million so far,” says Chattopadhyay.
The open source solutions also simplified management and reduced systems maintenance costs. “Even though the server count has increased by more than twenty times (20x), we manage the infrastructure with almost the same team size,” says Chattopadhyay.
Further, the combination of Dell Power Edge Servers and Dell desktops with Red Hat proved to be a pocket friendly deal. “The Dell servers fitted our specific requirements with no frills. This cut costs in terms of hardware specs. Since we had compiled Ubuntu only for specific tasks, we needed lower spec hardware to run it, compared to what was needed to run generic Ubuntu software. Dell allowed us to do that through its flexibility in the desktop and server segment,” says Chattopadhyay.
But, with all the in-house innovation done by Just Dial’s IT team, did the development costs dilute the overall cost advantage somewhere down the lane? Just Dial believes that when working on something fundamentally core to its business proposition, sometimes what is viewed as an outgoing cost could be what builds immense value for the firm. So Just Dial chose to invest heavily in customising open source, confident that it would turn into a good investment.
“We wanted to be different and were ready to absorb some costs, which also included time. We believed in getting it right the first time and in constant improvement. Our revenue growth speaks of our success in changing this cost centre into an investment. Despite the money we have put in to changing pathways, we have grown our revenues by two times after moving to open source,” says Chattopadhyay.
Improved customer satisfaction and flexibility
The open source architecture provided greater flexibility, which in turn led to indirect improvements in customer satisfaction. “Open source has given us great flexibility. For instance, if I got a great algorithm for a map search, I am confident that it could be made to work on the Red Hat platform. Such flexibility allowed us to experiment, develop and opt for tools and algorithms to optimise our data structures. The proficiency we achieved led to an indirect increase in efficiency and customer satisfaction. Besides this, discussions with the community opened up new horizons helping us come up with better products for our customers,” says Chattopadhyay.
Perfect alignment with growth vision
Open source fuelled Just Dial’s meteoric growth. “We were in just seven cities in 2007. Only when we switched to open source did we see the chance to grow to 240 cities on a single national number — 69999999. Open source technology aligned with and fostered our vision of growth. It made us realise we could grow at a much faster rate,” says Chattopadhyay.
Looking ahead, open source will continue to be the force powering Just Dial’s ambitious plans to expand outside India. “We have already launched our services in the US, and in the near future we will expand to most of the English speaking countries. The way we are as a company — impulsive and focused on the market — only open source can keep up with us,” says Chattopadhyay.
Easy maintenance
Maintenance of such an infrastructure could have been a nightmare, especially with all the innovation embedded into it. Red Hat support, along with Just Dial’s fully equipped IT team, helped the company address this challenge without too much trouble. “Red Hat support, along with our IT team of about 40 people on the infrastructure side (managing Linux and the servers) and another 90 people on the development side (working on database management, core C programming, the user interface, and backend page scripting) helped us maintain our infrastructure with great ease,” says Chattopadhyay.
The relationship just seems to get stronger
Looking ahead, open source technology will continue to be Just Dial’s first love. The firm has plans to opt for an open source application server soon. “Since we have consolidated and arrived at a convergence of the modules and IT requirements, we are looking for an open source application server with tidy coding and a plan to build around it. We have benchmarked one, called Yii, that works with any C component, making it a perfect fit for us. Yii’s memory management for connection pooling of databases promises to give us a big advantage. It also has a lot of fault tolerance and sharing. Lastly, Yii has good community backing; it has grown in the right fashion with a loyal fan base,” says Chattopadhyay.
With Fedora coming under the Red Hat umbrella and having rectified its network stack issues, Just Dial plans to experiment with Fedora 13. “Also, with the fate of MySQL shaky, the company is looking at other DBMS options like Flash DB, Percona, MariaDB, etc,” says Chattopadhyay. With such achievements to their joint credit, Just Dial will remain an ardent open source fan and user.
