‘Every open source project at Facebook is a little different’

Facebook is already leading the world of social networks. Today, it attracts extensive open source contributions from around the world. James Pearce, open source head at Facebook, highlighted some of the major projects run by the social networking giant in an exclusive talk with Jagmeet Singh of OSFY. Edited excerpts…

Q Why did Facebook opt for open source?
Open source is an important part of our engineering culture, and this has been the case ever since the site was first built on platforms like Apache, PHP and MySQL back in 2004. Since then, we have learned to share our work as responsible stewards of projects like HHVM, React, React Native and hundreds of others.

We see lots of advantages in bringing the community and industry together to openly share ideas about how to solve new engineering problems caused by scale, how to innovate faster, and how to share insights into the sort of technical challenges that Facebook needs to address.

Q In what ways is the Open Compute Project crucial for developing regions like India?
The Open Compute Project (OCP) has allowed us to take the same kind of creativity and collaboration we see in open source software to the hardware space. Through the OCP community, we have been able to break open the traditional black box of proprietary infrastructure, and foster a collaborative and open community around the world. The project provides a structure for individuals and organisations to share designs with others and help the data centre and hardware industries evolve.

By openly sharing ideas and specifications, we can reduce complexities in tech components that limit our ability to innovate more quickly. This approach is beneficial to everyone because we can leverage more choices to achieve better flexibility in how we build hardware and software in the data centre.

Q How did you build a community to handle open source developments?
There is no one-size-fits-all. Every open source project at Facebook is a little different, and the way we work with various communities varies (both technically and geographically). Some have active meet-up events, while some have regular blog posts. Others focus solely on a specific project to work through issues and pull requests that have come in.

My team is here to make sure that the people on each project think about this and do the job well — and that the best techniques for community engagement are shared. There are definitely several things that are common across all our active projects.

First, we make sure that our open source projects are well aligned with the work that we do. So, for example, the commits our engineers make are being shared to GitHub in real-time, and the community can follow along as our work happens. This also means that incoming contributions are accepted quickly without merge issues.

Second, we work hard to support the early adopters of our projects, giving them advice, pointers to possible areas of contribution, and lots of recognition. This first wave of community membership then goes on to help us advocate the next people that want to be involved.

And finally, we always try to celebrate the work that community members or other companies do, whether large or small. Sometimes the most important thing about a community is to know about all the other things that are happening in it!

Q Where does India stand in terms of open source contributors?
India is an important country for Facebook, and the largest country for developers building with Facebook outside of the US. With more than 70 per cent of Facebook’s developer ecosystem based outside of the US, our developer community is bigger and more global than ever.

With 1.7 billion people on Facebook and more coming online every day, we want to help developers around the world build engaging, useful experiences and services that make people’s lives better. One of the ways we help developers grow is with free tools and services to either get their businesses up and running or take these to the next stage of growth with our FbStart program. Through openness and collaboration around the world, developers can quickly build businesses that are global and scalable.

Q Do you consider Indian developers as capable of building interesting open source solutions as the developers in the US and other developed markets?
Absolutely! One of the many benefits of open source is that anyone in the world can have access to and share ideas and resources, so that we can solve more complex technical challenges than we would on our own.

It is fascinating to see how companies of all sizes and in various industries can quickly discover more creative ways to address new engineering challenges through openness and collaboration. As more people come online, the open source community is growing to more regions across the globe, which means there is an opportunity to make the world more open and connected through two-way dialogues in code. On an open source project, sometimes you never even know where in the world your community and collaborators are actually living!

Q Any particular inspiration behind Facebook’s open source adoption?
Open source is a catalyst for faster development. But, at the same time, it is also the right thing to do. At Facebook, our mission is to make the world more open and connected, and we believe this approach applies to the craft of developing software just as much as it does to connecting people on Facebook itself.

By making our projects open, developers can share ideas across companies and work on the code together, thus speeding up the development process. Through open source, new technology becomes available faster than the traditional market would allow, and this helps the entire industry address many of today’s complex technical issues together.

There is a passion for sharing that drives a lot of the work our teams here do. It also inspires us to find new ways to build and evolve those teams.

Q What challenges did you face in finding appropriate talent?
The great thing that we found about open source is that it actually is a huge help when searching for the right talent and in building strong connections with engineers that share our ambitions and like solving the same problems as us. Since the people working on our open source projects are the same engineers that are building our production-grade technology, engineers interested in working at Facebook already have a good sense of how the team operates, and vice versa.

We also conduct a survey when new employees join the company, and one of the questions we ask is whether open source was one of the deciding factors for them coming to work at Facebook. We are always proud when we hear it has been!

Q Is India already a source of talent for various open source developments, or do Indian developers still need to acquire some vital skills to join the team under your leadership?
APAC is one of the fastest-growing regions for developers building with Facebook, outside of the US. So developer skillsets in the area are critical to Facebook. As more people come online every day, we want to help developer communities grow to more regions across the globe. To help with this, we created FbStart, a program designed to help early stage mobile startups build and grow their applications, which now has more than 9,000 members spanning 137 countries. India provides the most FbStart members in APAC.

Q Do you think that open source is efficient enough for a social network like Facebook?
The key to ensuring that our projects are easy to use and work with externally is to make them well-architected, modular, and with clean APIs. This means we must write better software in the first place, and then make our own code-reuse and maintainability much better.

It is also important to continuously improve the documentation for each open source project. Proper documentation helps ensure that those who are unfamiliar with the technology can adopt it quickly. This also helps when new engineers inside the company want to ramp up on a project quickly.

Q How do you choose the open and closed source mix?
This is the question we constantly revisit. It can be easy for companies to turn their open source portfolios into a dumping ground sometimes — meaning that code is uploaded once but not maintained. This is often a result of not having the right tools and program management in place.
At Facebook, we want to grow the open source community and to ensure we are moving forward, we make sure that every project we plan to open source will be well maintained and managed. This is one major criterion, but we also tend to focus on projects that we think offer something new and useful for the industry rather than just reinventing the wheel, for the sake of it.

It is worth pointing out that we also have a shutdown process for projects that are no longer valid or well maintained. This allows us to be clear about their status without stopping the community from taking them over and further developing them.

Q Don’t you think that deploying open source comes with some serious security concerns? How would you overcome all this?
Openness and security are not mutually exclusive. In fact, open source generally makes your code more secure. By making the code open to the public, developers can help spot vulnerabilities and bugs in your code that you might have otherwise missed.

Q Facebook has now emerged as the leading artificial intelligence (AI) company. How can open source take concepts like AI and deep learning to new levels?
We are excited to pursue projects in many areas and industries beyond traditional developer platforms, including artificial intelligence and machine learning. The industry is starting to recognise that these areas can help us leverage the power of automation to build more quickly and improve the overall performance of our products. Through the power of artificial intelligence and machine learning, we can think more creatively about how we can continue to accelerate the pace of innovation and build breakthrough technology.

And we are applying many of these concepts into our own software at Facebook. For example, we open sourced Torch implementations and training scripts of deep neural nets for image recognition as well as Torchnet — a modular collection of boilerplate code, key abstractions, and reference implementations that build on top of the Torch framework.

We are eager to see how intelligent tooling can help engineers and developers work more efficiently than they do today, at scale.

Q So can we say that open source is the key to Facebook’s success?
There are so many things that contribute to Facebook’s success, but we are certainly proud to have been able to build alongside — and play our part in contributing back to — the open source movement. Here is to continuing to do so!