“AI (artificial intelligence) is going to change the world more than anything in the history of mankind. More than electricity,” said Dr Kai Fu Lee, AI oracle and venture capitalist. His words ring true today, as AI pervades almost every part of human life. This article talks about how chatbots can be built using the Rasa framework.
All of us use artificial intelligence and machine learning (AIML) technology in some way or the other today — be it video predictions, online shopping, social media, surveillance, food delivery, transportation, self-driving cars, and so on.
National language processing (NLP) is a discipline of AI that is used to help understand, interpret and manipulate human language. Let us first look at NLP and its applications, and then build a chatbot from scratch using the Rasa framework.
Natural language processing
NLP helps computers communicate with humans in their own language and scales other language-related tasks. For example, it opens up the possibility for computers to read text, hear and interpret speech, as well as measure sentiment and determine important parts of it.
Today’s machines can analyse more language based data than humans, without fatigue and in a consistent, unbiased way. Considering the staggering amount of unstructured data, from medical records to social media, that’s generated every day, automation is critical to fully analyse text and speech data efficiently.
Sample applications of NLP include sentiment analysis, customer reviews, customer segmentation, anomaly detection, product improvement; topic modelling such as coming up with new topics from the text and using them to assign new supervised learning labels, providing insights that are too difficult to find from manual searching; and text categorisation of animal species, fake news, bank transactions, etc.
Chatbots
A chatbot is a computer program that simulates human conversation through voice commands, text chats or both. Short for chatterbot, it is an AI feature that can be embedded and used through any major messaging/text application. This automated program interacts like a human, and costs little to nothing to engage with. Key examples are those used by businesses in Facebook Messenger, or as virtual assistants, such as Amazon’s Alexa.
Chatbots tend to operate in one of two ways—either via machine learning or with set guidelines.
Rasa
Rasa helps in creating virtual assistants, which are used to automate human-to-computer interactions across platforms ranging from websites to social media. It supplies conversational AI infrastructure for a global community of developers, providing the tools to build chat-based and voice-based contextual assistants.
Rasa is powered by open source software and runs across organisations ranging from startups to Fortune 500s, and industries like healthcare, financial services, retail and insurance. It has three main functions, which together provide everything needed to build a virtual assistant.
Natural language understanding: It converts raw text from user messages into structured data, parses the user’s intent and extracts important key details.
Dialogue management: Machine learning-powered dialogue management decides what the assistant should do next, based on the user’s message and context from the conversation.
Integrations: It has built-in integration points for over 10 messaging channels, plus endpoints to connect with databases, APIs, and other data sources.
Installation
The first step is to install Rasa.
Installation The first step is to install Rasa. % python -m venv env % source env/bin/activate % pip install rasa For UI: pip install rasa-x -i https://pypi.rasa.com/simple [optional] % rasa init # setup the basic file structure % update config.yml and endpoint.yml [optional] % rasa train # train your model % rasa train nlu % rasa train core % rasa shell # run the bot from shell % rasa run actions # action server % rasa data validate # validate if the data given in the configuration is good % rasa run -m models --enable-api --cors “*” --debug # run the rasa as rest http server
A sample UI installation (not associated with Rasa) is given here. To have conversations with Rasa in the user interface (UI), there is a module called chatroom. It is ReactJS based (URL: https://github.com/scalableminds/chatroom).
% mkdir ui; cd ui % git clone https://github.com/scalableminds/chatroom.git % cd chatroom % yarn install % yarn build (after any customised changes) % yarn serve # to run the ui server % update the index.html with the address of the rasa server running
Open the URL http://127.0.0.1:8080/index.html in Chrome, as seen in Figure 1. (Note: Chrome is the supported browser for now.)
Figure 2 shows the sample conversation, which was written from scratch using Rasa.
Now that we have seen the conversation, let’s see how each scenario is written step by step using the Rasa framework. We will also gradually learn the concepts of Rasa from these scenarios.
For each scenario we will see the output in UI form.
Scenario 1
College timings
nlu.yml
- intent: timings examples: | - I would like to college timings - college time please - what would the class start and end time - when college reopen
stories.yml
- story: college time path steps: - intent: timings - action: utter_timings
domain.yml
intents: - timings responses: utter_timings: - text: “The college is not open now we are still working through online”
Description
- rasa run
rasa train: Need to run this after every change we do in the configuration files.
rasa nlu train: This is run if only the nlu is updated, such as nlu.yml, stories.yml and rules.yml.Once the train is completed, the model will be saved under model/directory. For example, you will get the message: Your Rasa model is trained and saved at ‘/Users/dev/Desktop/Technical/Blogs/Chatbot/rasa/models/20210925-140501.tar.gz’ - nlu.yml
NLU (Natural Language Understanding) is used to store the training data and extract structured information from user messages. This usually includes the users’ intent and any entities their message contains. From the above example, the student can know the college timings; so give some sample input for the bot to learn the student’s intention.
- stories.yml
Stories help the bot to learn dialogue management. They can be used to train models that are able to generalise unseen conversation paths. From the above example, when the user’s intention is to know the college timings, the bot will respond to the student’s message, saying the “colleges are still operating online.” - domain.yml
Domain is the key file for the Rasa framework. It specifies the intents, entities, slots, responses, forms, and actions your bot should know about. It also defines a configuration for conversation sessions. From the above example, we have specified the intents and our responses here. The output in UI can be seen in Figure 3.
Scenario 2
Course duration for department
nlu.yml
- lookup: department
examples: |
- civil
- mechanical
- computer
- textile
- printing
- intent: course_duration
examples: |
- what is course duration for [civil]{“entity”: “department”}
- would like to know the [computer]{“entity”: “department”} course tenure
stories.yml
- story: college course duration steps: - intent: course_duration - action: action_course_duration
domain.yml
intents:
- course_duration
actions:
- action_course_duration
entities:
- department
responses:
utter_course_duration:
- text: “For {department} course is of {duration} months”
actions.py
class ActionCourseDuration(Action):
def name(self) -> Text:
return “action_course_duration”
def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
# It will return array of entities
entities = tracker.latest_message[‘entities’]
print(entities)
course_duration = {
‘civil’: 10,
‘computer’: 12,
‘mechanical’: 14,
‘printing’: 16,
‘textile’: 18
}
entity_department = None
# Iterating through the array to retrieve the desired entity
for e in entities:
if e[‘entity’] == “department”:
entity_department = str(e[‘value’]).lower().strip()
duration = course_duration.get(entity_department, 0)
dispatcher.utter_message(
response=”utter_course_duration”,
department=entity_department,
duration=duration
)
return []
Description
- nlu.yml
Here we can see something new, known as entity and lookup tables. Entity is the information extracted from the intent. Lookup tables are lists of words that help to extract entities. Here we are defining a list of departments/courses offered in the college, as it will help the bot to reply better based on the student’s department. - stories.yml
When the user asks about the course duration, the reply is not just a plain spoken message; instead, it’s an action taken. - actions.py
The actions file holds the custom action that can run any code you want. This is used to make an API call, or to query a database, for example. Rasa framework has a beautifully decoupled actions server; to run it, we need to call ‘rasa run actions’. From the example, we are returning the course duration saved from the dict, but it could be anywhere.Output in the UI for this scenario is seen in Figure 4.
Scenario 3
Exam results
nlu.yml
- regex: rollnumber
examples: |
- \d{10,30}
- intent: get_roll_number
examples: |
- my roll number is [1234567891](rollnumber)
- This is my roll number [1234567891](rollnumber)
- [1234567891](rollnumber)
- intent: request_result
examples: |
- may I know the exam results
- can you please help me to know if I have passed
- am I all clear
rules.yml
- rule: activate result form steps: - intent: request_result # intent that triggers form activation - action: result_form # run the form - active_loop: result_form # this form is active - rule: submit form condition: - active_loop: result_form # this form must be active steps: - action: result_form # run the form - active_loop: null # the form is no longer active because it has been filled - action: utter_submit # action to take after the form is complete - action: utter_slots_values # action to take after the form is complete - action: action_show_result
domain.yml
intents:
- get_roll_number
- request_result
actions:
- action_show_result
forms:
result_form:
required_slots:
rollnumber:
- type: from_entity
entity: rollnumber
slots:
rollnumber:
type: any
entities:
- rollnumber
responses:
utter_result:
- text: “For {roll}, result is {result} with {score} score”
utter_ask_rollnumber:
- text: “Please provide your roll number”
utter_submit:
- text: “All done!”
utter_slots_values:
- text: “I am going to run a result search using the following parameters:\n
rollnumber: {rollnumber}”
actions.py
intents:
- get_roll_number
- request_result
actions:
- action_show_result
forms:
result_form:
required_slots:
rollnumber:
- type: from_entity
entity: rollnumber
slots:
rollnumber:
type: any
entities:
- rollnumber
responses:
utter_result:
- text: “For {roll}, result is {result} with {score} score”
utter_ask_rollnumber:
- text: “Please provide your roll number”
utter_submit:
- text: “All done!”
utter_slots_values:
- text: “I am going to run a result search using the following parameters:\n
rollnumber: {rollnumber}”
Description
- nlu.yml
Sometimes, it can be hard to configure all the possible values. Regex comes to the rescue here. In this scenario, we can see Regex being used to identify the roll number. - rules.yml
Rules are a type of training data used to train your assistant’s dialogue management model. They describe short pieces of conversations that should always follow the same path. Forms are used to save the student’s roll number. The most common conversation patterns are to collect a few pieces of information from a user in order to do something (book a restaurant, call an API, search a database, etc).
| Note: Don’t overuse rules. Rules are great to handle small specific conversation patterns, but unlike stories, they don’t have the power to generalise to unseen conversation paths. |
- domain.yml
Slots are your bot’s memory. They act as a key-value store, which can be used to store information the user provided (e.g., their home city) as well as information gathered about the outside world (e.g., the result of a database query). Here we have saved the student’s roll number in a slot and it is extracted from the entity.The output for this scenario in UI is seen in Figure 5.
Scenario 4
- Fee enquiry
- Provide if only roll number is provided
- Else ask for the roll number and then provide the fee structure
nlu.yml
- regex: rollnumber
examples: |
- \d{10,30}
- intent: get_roll_number
examples: |
- my roll number is [1234567891](rollnumber)
- This is my roll number [1234567891](rollnumber)
- [1234567891](rollnumber)
- intent: fees_enquiry
examples: |
- may I know the fees structure
- how much fees do I need to pay
- do I have any pending fees to be paid
stories.yml
- story: Ask for rollnumber and say fees steps: - intent: fees_enquiry - slot_was_set: - rollnumber_provided: null - action: utter_ask_rollnumber - intent: get_roll_number - slot_was_set: - rollnumber_provided: true - action: action_save_roll_number - action: action_fees_details
rules.yml
- rule: Only say `fees` if the user provided a rollnumber condition: - slot_was_set: - rollnumber: true steps: - intent: fees_enquiry - action: action_fees_details
domain.yml
intents:
- fees_enquiry
actions:
- action_fees_details
entities:
- department
- rollnumber
slots:
rollnumber:
type: any
responses:
utter_fees:
- text: “For {roll}, fees is {fees} INR.”
actions.py
class ActionShowFeesStructure(Action):
def name(self) -> Text:
return “action_fees_details”
def run(self,
dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
roll = tracker.get_slot(“rollnumber”)
print(“Rollno: “, roll)
fees = 0
if( roll ):
fees = 10000
dispatcher.utter_message(
response=”utter_fees”,
fees=fees,
roll=roll
)
return []
class ActionReceiveRollNumber(Action):
def name(self) -> Text:
return “action_save_roll_number”
def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
#text = tracker.latest_message[‘text’]
entities = tracker.latest_message[‘entities’]
roll = None
for e in entities:
if e[‘entity’] == “rollnumber”:
roll = str(e[‘value’]).lower().strip()
dispatcher.utter_message(text=f”I’ll remember your rollnumber {roll}!”)
return [SlotSet(“rollnumber”, roll), SlotSet(“rollnumber_provided”, True)]
All the configurations we have learnt so far will remain the same. But this scenario is written to test the bot’s memory (slot). Here, the student is not required to enter the roll number again. And in the story we have written, the roll number will be asked for if the student has not provided it, and only then will the fee structure be provided. The output is seen in Figure 6.
Scenario 5
Change of department request
nlu.yml
- intent: department_have_been_changed
examples: |
- I have changed from [civil]{“entity”: “department”, “role”: “from”}
- Have moved from [civil]{“entity”: “department”, “role”: “from”}
- intent: department_going_to_change
examples: |
- I am going to [civil]{“entity”: “department”, “role”: “to”} department
- I am changing to [civil]{“entity”: “department”, “role”: “to”} department
- Will be moving to [civil]{“entity”: “department”, “role”: “to”} course
stories.yml
- story: The student moving from another department steps: - intent: department_have_been_changed entities: - department: Civil role: from - action: utter_ask_about_experience - story: The student is going to another department steps: - intent: department_going_to_change entities: - department: Computer role: to - action: utter_wish_luck
domain.yml
intents: - department_have_been_changed - department_going_to_change responses: utter_ask_about_experience: - text: “How was your experience with the department.” utter_wish_luck: - text: “Wish you best luck in the new department.”
Description
- nlu.yml
Here we are using the features Entity Roles and Groups, where we need to specify the list that the roles and groups of an entity can belong to.
Figure 7 gives the output in UI.
Rasa can help us build a chatbot for use cases quickly. The scenarios given here are pretty basic, and there is a lot more that Rasa has to offer.
