Chatbot Architecture: How to Build a Chatbot

2 June, 2020

Chatbots boost your business processes and streamline customer support operations. Knowing the underlying concepts of chatbot development helps find the right team for the job.

Machine learning is competing with human capabilities on so many levels that it is no surprise chatbot systems are thriving. A text bot uses artificial intelligence technologies designed to assist customer support systems, perform voice searches or simply provide a companion to chat with. At the backend of these systems, various approaches are used to provide correct answers to queries from the user.

The text bot is provided with the question and the domain knowledge from all the sources. It has to interpret these and find the correct answer to the questions with precision & accuracy, correct syntax & semantics and in the least possible time.

A scheme illustrating the insides of user-chatbot communication pattern

Wondering how to build a chatbot? We would say that there’s no rocket science involved. In order to build a chatbot that is open domain and AI-based, we must go through various steps to perform query analysis and answer generation. Each step involves natural language processing (NLP) and machine learning algorithms to get the clue which will help us get the final answer. The basic approaches and methods used in chatbot development are described further below.

Lexical Answer Type Chatbot Architecture

Lexical Answer Type (LAT) prediction is the process of assigning certain answer types to each user question. This works on the principle of first finding the clue of the LAT in question and then finding out the type of the clue.

This type of chatbot architecture was used in the Watson model. It proves to be very efficient as for every question the data scientists are able to find out the clue and optimize search performance based on that clue to reduce the domain. For example, here’s an answer generated by an LAT-based text bot:

Input: “maneuver”

Category: Oooh….Chess

Clue: Invented in the 1500s to speed up the game, this maneuver involves two pieces of the same color.

Answer: Castling

Steps Taken to Convert Data to Knowledge

Domain knowledge is converted to indices and passages for lookup. Using these passages the model is trained with the previously available data beforehand. The model cuts down important frames from the domain knowledge and stores just the likely context of it. These frames are stored in a file to be used as the key in answering the question if the clue value is related to it.

Training and test question sets used for the pre-training of the machine. A set of questions is made and is used to train the model by using logistic regression or any other algorithm. The model is thus trained with some data that can be used as knowledge for the machine.

NLP resources are used for setting up grammar and vocabulary. Vocabulary is used for creating relationships among the named entities and for detection purposes as well. The NLP operations needed to understand the query and obtain useful information from it are described further.


Tokenization is used for chopping up the input into pieces called tokens and throwing away unnecessary characters such as punctuation. To put it simply, this is meant to split apart the text into individual units, while each individual unit has a value associated with it.

Here’s an example of how we can do this using the Natural Language Toolkit:

import nltk
sentence_data = "The First sentence is about Python. The Second: about Django. You can learn Python, Django and Data Analysis here."
nltk_tokens = nltk.sent_tokenize(sentence_data)

This will create a subdivision among the sentences to create subproblems. From the example above, the code will create two units:

  • “The first sentence is about Python”.
  • “The second about Django”.

Named Entity Representation

This process involves spotting named entities (people, places, organizations etc.) from a chunk of text, and classifying them into a predefined set of categories. Here’s an example implemented using spaCy library:

import spacy

nlp = spacy.load('en_core_web_sm')
sentence = "Apple is looking at buying U.K. startup for $1 billion"
doc = nlp(sentence)

for ent in doc.ents:
    print(ent.text, ent.start_char, ent.end_char, ent.label_)

This will output the inputted sentence into various categories like these:

Apple 0 5 ORG Apple is the entity here; 0 and 5 are coordinates within a sentence; ORG is the label for the category.
U.K. 27 31 GPE U.K. is the entity here; 27 and 31 are coordinates within a sentence; GPE is the label for the category.
$1 billion 44 54 MONEY $1 billion is the entity here; 44 and 54 are coordinates within a sentence; MONEY is the label for the category.

LAT Combined with Focus Detection

LAT can be used in combination with focus detection to find clues from the given question and generate a simpler form of the question. An answer can be then given to that question. See an example below:

Question: Can arctic animals be invisible?

Prediction: Can polar bears be seen under infrared?

Answer: Polar bears are invisible under infrared.

Semantics Extraction

This process involves building structures that approximate concepts from a large set of documents. It generally does not require prior understanding of the documents. Different APIs like auto-tagging of text and part of speech tagging using open NLP are used for this purpose.

Pronoun Resolution

Pronoun Resolution helps determine the pronouns in the sentence and creates entities for each pronoun. After that it attributes information to these entities and classifies them as follows:

Original text: Tom and Jane are good friends. They are cool. He knows a lot of things and so does she. His car is red, but hers is blue. It is older than hers. The big cat ate his dinner.

Pronoun resolution: Tom and Jane are good friends. Tom and Jane are cool. Tom knows a lot of things and so does Jane. Tom’s car is red, but Jane’s is blue. His car is older than Jane’s. The big cat ate The big cat’s dinner.

In the code example below we use the StanfordCoreNLP toolkit for the chatbot app implementation:

from pycorenlp import StanfordCoreNLP

nlp = StanfordCoreNLP('http://localhost:9000')

def resolve(corenlp_output):
    for coref in corenlp_output['corefs']:
        mentions = corenlp_output['corefs'][coref]
    antecedent = mentions[0]  # the antecedent is the first mention in the coreference chain
    for j in range(1, len(mentions)):
        mention = mentions[j]
        if mention['type'] == 'PRONOMINAL':# get the attributes of the target mention in  the corresponding sentence
            target_sentence = mention['sentNum']
            target_token = mention['startIndex'] - 1
            corenlp_output['sentences'][target_sentence-1]['tokens'][target_token]['word'] = antecedent['text']

Content Preprocessing and Evidence Analysis Models

Context preprocessing creates the frames from content like subject-verb objects (SVO) which cuts the frame containing the SVO occurrences in the text. Evidence analysis uses LAT detection and named entity alignment-based scoring. The latter is the major component in the system.

Automated Learning From Reading

These types of chatbot architecture use books or relevant information and convert it into syntactic frames using sentence parsers. These syntactic frames are then converted to semantics using generalization and are latered stored as the training data.

Keyword Evidence-Based Chatbot Design

A Keyword evidence-based chatbot app is trained to be able to find the keywords in the input received and the knowledge stored in the system. For example:

Input: In May 1988, Portugal celebrated the anniversary of this explorer that arrived in India.

Knowledge stored in memory: In May, Gary arrived in India after he celebrated his anniversary in Portugal.

The chatbot model here must be able to find the answer as Gary and tag the other keywords in both sentences as well.

Combining Methods to Generate More Precise Answers

All the methods described above can be used in combination to get an answer from the complex corpus we have. By applying all the clues to the corpus, we try to reach the best possible answer for the dataset. Paragraphs and sentences with the best match for all of the available clues are given the highest confidence score. The top three can be given as the final answer. Deeper evidence that brings more precise results can become much harder to find as it includes hypotheses from various sources.

How IBM used DeepQA to Build a Chatbot

Two human competing Watson in Jeopardy game

To understand the process of developing AI-based chatbot architecture, let’s take an example of a real-world implementation. IBM Watson, for instance, was based on DeepQA architecture using NLP and machine learning.

DeepQA Chatbot Architecture

Our natural language is hard to understand by computers. Thus, in order to create a chatbot architecture that works with user input, we need to make this language interpretable by the machines. DEEPQA is basically the software architecture for deep content analysis and evidence-based reasoning to let the computer learn from user inputs.

It has a very high level of capability that uses advanced natural language processing, semantic analysis, information retrieval, automated reasoning, and machine learning. The DeepQA analyzes the natural language input and generates relevant answers by unifying the knowledge available and existing natural language text databases.

The DeepQA architecture system views the problem of Automatic Question Answering as a parallel hypothesis generation and evaluation task. Thus, it is not just about the question-in and answer-out, rather it is about the various possibilities of the hypothesis based on relevant key evidence.

With a question topic or set of questions, DeepQA finds out relations and useful concepts from the question, building a representation of the user’s information. After that, it generates possible responses via searching for the answer within this chunk of data.

Finally, it delivers the rank list of each of the hypotheses along with the score for each of these to be used further to generate the answer. Thus, many different tasks are used to lead the automated question answering, including domain-specific knowledge.

IBM Watson Chatbot Design Specifics

Scheme of IBM Watson platform working process

DeepQA architecture was responsible for most of the work done by IBM Watson. It was used for generating hypotheses, gathering massive evidence, and analyzing the data. The DeepQA team’s implementation used the following methods to build a chatbot:

Content Acquisition

Content Acquisition was done manually but also included automated steps. Analyzing example questions is a manual task while domain analysis is performed using automatic or statistical analyses, such as the LAT analysis.

The first step is to analyze example questions from the problem space to produce a description of the kinds of questions that must be answered which is referred to as answer type detection. This is a fundamental step before moving towards further answering.

Question Analysis

Question Analysis is the process of understanding the user query. There are multiple steps involved in performing Question Analysis which include various NLP functions like tokenizing the question, entity extraction, focus detection. All of these operations will be helpful as a clue to generate the answer.

Moreover, the questions are classified on the basis of the type of problem being asked,  like logical puzzles or computations, etc. Using LAT to find clues from the question to start with the processing are all part of the question analysis.

Hypothesis Search

Hypothesis generation is performed in several steps. The primary search uses a keyword evidence-based search and considers the top 200 instances of the answer received. Analysis shows that most of the correct answers come from these 200 samples. Thus, candidate answers (CAs) are generated using these primary searches.

Filtering of the primary samples is done using some models to decrease the number of samples from the primary search. Hence, if the answer is not valid from the decreased set then it further uses the leftover answers from the primary searches. Each CA is then put back into the question to be used as the hypothesis for the answer. If there is a failure at this stage, the system considers and recalls the other pipeline and works again with that pipeline.

Evaluating the Generated CAs

The next task is to generate the support vector that is the evidence for the support of each answer. This can be done using passage search or evidence generated during the keyword evidence search which is used to provide the evaluation criteria for each CA. The score for each of the candidate answers is then evaluated and each of them is given a rank which increases the probability of getting access to the correct answer early on in the process.

Final Merging

In this step, merging of the hypothesis is carried out after the evaluating score. It is done as answers from various platforms may be equivalent to be produced as an output. An ensemble of matching normalization and co-reference algorithms are used, which helps in the selection from the equivalent answers.


The final step is to rank all the hypotheses and build the confidence score for each of them. Now, these hypotheses are run over the trained ML model to find answers for the questions. The models are trained on the predefined hypothesis generated from the present domain knowledge and all the data available at the time of the model training.This model generates prediction results to provide the answer to the question asked.


Chatbot development methods need to be chosen based on the demands of the client. For the most part, it’s easier to go with NLP and ML-based architecture. Depending upon the requirement, resource availability, and installed infrastructure, domain-specific chatbots will require a combination of LAT, focus detection, automated learning, content preprocessing, and evidence analysis models. Some of the use cases do not require complex chatbot architecture, so companies looking for implementation are advised to consult with a qualified AI development company.


PixelPlex AI development company boasts a squad of machine learning solutions engineers, data science experts, and other AI software development pros. Reach out to us — we’ll help you translate big data or disparate digital assets into business growth triggers.

Learn more
Suite 31520 West 28th St.New York, NY 10001
More contacts


PixelPlex AI development company boasts a squad of machine learning solutions engineers, data science experts, and other AI software development pros. Reach out to us — we’ll help you translate big data or disparate digital assets into business growth triggers.

Learn more