- Facebook said that Blender is the first chatbot to build a diverse set of conversational skills like empathy, knowledge, and personality in one system
- The tech giant also said the bot uses transformer neural networks on large amounts of conversational data
Facebook’s AI has built an open source open-domain chatbot called Blender as per a blog post by the company. Engadget reported that it has been trained on 9.4 billion parameters and more than 10 times as many as the previous largest OS chatbot available on the internet.
Facebook said that Blender is the first chatbot to build a diverse set of conversational skills like empathy, knowledge, and personality in one system. The blog stated that the bot in terms of engagement feels “more human” according to human evaluators. It has been designed in such a manner that it can assume a persona, discuss nearly any topic, and show empathy in natural, 14-turn conversation flows.
Transformer neural networks
The tech giant also said the bot uses transformer neural networks on large amounts of conversational data. It uses previously available public domain conversations that have 1.5 billion training examples of extracted conversations.
Facebook said that its neural networks are too large to fit on a single device. Hence, it utilised techniques such as column-wise model parallelism. This allowed splitting the neural network into smaller, more manageable pieces while maintaining maximum efficiency. Facebook also said that to make sure conversational agents do not repeat themselves or display other shortcomings, researchers typically used a number of possible generation strategies after the model is trained.
Blended Skill Talk
Stephen Roller, a research engineer at Facebook’s AI lab (FAIR) told Engadget that there are a lot of sophisticated techniques that can be used in terms of how one can chop this thing up. If one splits it over different devices and chop it up like the wrong way, then one can lose that efficiency and not be able to scale to the terabyte-sized data sets that the company been working with.
Facebook said that it has introduced Blended Skill Talk (BST) for training and evaluating desirable skills such as personality, knowledge, and display of empathy. In its composition, it also has beam search, token sampling, and n-gram blocking.
