Facebook deployed Horizon over the past year to improve platform’s ability to adapt RL’s decision-based approach to large-scale applications
Facebook announced in a blog post about open sourcing its software Horizon, its code is now available on GitHub. Facebook was the first company to make what it calls “an end-to-end”’ reinforcement learning program designed for addressing large-scale business problems freely available, stated by Jason Gauci, a Facebook Engineer who worked on Horizon.
Facebook used Horizon internally to optimize how 360-degree videos are displayed on the social network, taking into account such factors as the available bandwidth and how much of the video has already been buffered. Now, anyone can download Horizon and use it to make some significant improvements in the social network’s video and notification features as well as its Messenger messaging app.
This AI tool was also used to improve what content to push to users through notifications. And it was used to hone the suggestions that its intelligence assistant, which is called M, makes to users of its Messenger app, – stated in the company’s bog post
Self Learning Platform: Horizon improves itself by trial-and-error from experience to maximize some reward or minimize some loss, rather than from labeled data sets. This AI tool contains many other features that make it safer to use reinforcement learning on real-world problems. For instance, the software helps programmers pick the right goal and rewards to feed the algorithm.
Rather than having algorithms start with zero knowledge and learn from random actions, Horizon initially trains algorithms to take a set of actions that a product engineer has specified. It then uses several kinds of counterfactual analysis, based on existing data, to simulate different actions the algorithm could have taken. In this way, Horizon mimics training the algorithm in a simulator, allowing it to be refined without worrying about it wreaking havoc in the real world.
Once the algorithm seems to be working well, Horizon allows users to carry out small-scale online experiments, using real data in real time, and then gradually roll the new algorithm out to larger sets of users or data. This entire process can then be repeated, with the fully-trained algorithm being used as the starting point for a new training series.
Facebook is also following other AI research groups: Alphabet Inc.’s DeepMind and GoogleBrain AI teams, and OpenAI, which have recently made reinforcement learning algorithms, programming tools and test environments publicly available.
“We’re committed to open source, so it was a natural decision to share this latest production-ready system for the community,” said by Srinivas Narayanan, the company’s director of applied machine learning.
Click here to see the blog post.
