In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-01-17 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > Development >
Share
Shulou(Shulou.com)06/02 Report--
This article focuses on "how to use Python to achieve chat robot project", interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "how to use Python to implement the chat robot project".
precondition
In order to implement a chat robot, a deep learning library Keras, a natural language processing toolkit NLTK, and some useful libraries will be used. Run the following command to ensure that all libraries are installed:
How does the pip installtensorflow keras pickle nltk chat robot work?
Chatbot is just an intelligent software that can interact and communicate with people like human beings. It's interesting, isn't it? Now let's see how they work.
All chatbots are based on the natural language processing (NLP) concept. NLP consists of two parts:
NLU (Natural language understanding): the ability of machines to understand human languages, such as English.
NLG (Natural language Generation): the ability of machines to generate text similar to human written sentences.
Imagine a user asking a chat robot a question: "Hey, what's the news today?"
The chatbot breaks down the user's statement into two parts: intention and entity. The purpose of this sentence may be to get news, because it refers to the action that the user wants to perform. The entity gives specific details about the intention, so "today" will be the entity. Therefore, a machine learning model is used to identify the intentions and entities of the chat.
Project file structure
All these documents will be left behind when the project is completed. Take a quick look at each one. It will give developers an idea of how to implement the project.
In this document, Train_chatbot.py- builds and trains a deep learning model that can classify and identify the requirements that users put forward to the robot.
The Gui_Chatbot.py- file is a place to build a graphical user interface to chat with trained chatbots.
The Intents.json-Intents file contains all the data that will be used to train the model. It contains a set of tags and their corresponding patterns and responses.
Chatbot_model.h6- this is a hierarchical data format file that stores the weight and architecture of the training model.
The Classes.pkl-pickle file can be used to store all the tag names to be classified when forecasting messages.
The Words.pkl-Words.pklpickle file contains all the unique words in the model vocabulary.
Download the source code and dataset:
Mailto: https://drive.google.com/drive/folders/1r6MrrdE8V0bWBxndGfJxJ4Om62dJ2OMP?usp=sharing
How to build your own chat robot?
The author simplifies the construction of this chat robot into five steps:
Step 1: import the library and load the data
Create a new python file and name it train_chatbot, then import all necessary modules. After that, read the JSON data file from the Python program.
Importnumpy as np fromkeras.models importSequential fromkeras.layers importDense, Activation,Dropout fromkeras.optimizers importSGD importrandom importnltk fromnltk.stem importWordNetLemmatizer lemmatizer = WordNetLemmatizer () importjson importpickle intents_file = open ('intents.json'). Read () intents= json.loads (intents_file)
Step 2: data preprocessing
The model could not get the original data. In order to make the machine easy to understand, it must go through a lot of preprocessing. For text data, there are many preprocessing techniques available. The first technique is tagging, which breaks down sentences into words.
By looking at the intents file, you can see that each tag contains a list of patterns and responses. Mark each pattern and add words to the list. In addition, create a list of classes and documents to add all the intentions related to the schema.
Words= [] classes= [] documents= [] ignore_letters = ['!','?','.] Forintent in intents ['intents']: forpattern in intent [' patterns']: # tokenize each word word= nltk.word_tokenize (pattern) words.extend (word) # add documents in the corpus documents.append ((word, intent ['tag'])) # add to our classes list ifintent [' tag'] notin classes: classes.append (intent ['tag']) print (documents)
Another technique is word form reduction. We can convert the words into Lemma form, so that we can reduce all the standard words. For example, the words play, playing, playing, played, and so on will all be replaced with play. This reduces the total number of words in the vocabulary. So take each word as a Lemma and get rid of the repeated words.
# lemmaztize and lower each word andremove duplicates words= [lemmatizer.lemmatize (w.lower () forw in words if w notinignore_letters] words= sorted (list (set (words) # sort classes classes= sorted (list (set (classes) # documents = combination betweenpatterns and intents print (len (documents), "documents") # classes= intents print (len (classes), "classes", classes) # words= all words, vocabulary print (len (words), "unique lemmatized words", words) pickle.dump (words,open ('words.pkl') 'wb')) pickle.dump (classes,open (' classes.pkl','wb'))
Finally, the word contains the vocabulary of the project, and the class contains all the entities to be classified. To save the python object in a file, use the pickle.dump () method. These documents will help to predict the chat after the training is completed.
Step 3: create a training set and a test set
In order to train the model, convert each input mode into a number. First, Lemma each word in the pattern and create a zero list that is the same length as the total number of words. Set the value 1 only to indexes that contain words in the pattern. Similarly, set 1 as the class input to which the pattern belongs to to create the output.
# create the training data training= [] # create empty array for the output output_empty = [0] * len (classes) # training set, bag of words for everysentence fordoc in documents: # initializing bag of words bag= [] # list of tokenized words for thepattern word_patterns = doc [0] # lemmatize each word-create baseword, in attempt to represent related words word_patterns = [lemmatizer.lemmatize (word.lower () for word in word_patterns)] # create the bag of words array with2 If word is found in current pattern forword in words: bag.append (1) if word inword_patterns else bag.append (0) # output is a'0' for each tag and '1'for current tag (for each pattern) output_row = list (output_empty) output_ row [classes.index (doc [1])] = 1 training.append ([bag, output_row]) # shuffle the features and make numpyarray random.shuffle (training) training= np.array (training) # create training and testing lists. X- patterns, Y-intents train_x= list (training [:, 0]) train_y= list (training [:, 1]) print ("Training data is created")
Step 4: train the model
The model will be a neural network composed of three dense layers. There are 128 neurons in the first layer and 64 in the second layer. The number of neurons in the last layer is the same as the number of classes. In order to reduce the overfitting of the model, the dropout layer is introduced. Use the SGD optimizer and fit the data to start the training of the model. After 200 stages of training, the Kerasmodel.save ("chatbot_model.h6") function is used to save the training model.
# deep neural networds model model= Sequential () model.add (Dense (128 len (train_x [0]), activation='relu')) model.add (Dropout (0.5)) model.add (Dense (64 ~ (10) model.add (Dropout (0.5)) model.add (Dense (len (train_y [0]) # Compiling model. SGD with Nesterovaccelerated gradient gives good results for this model sgd= SGD (lr=0.01,decay=1e-6, momentum=0.9, nesterov=True) model.compile (loss='categorical_crossentropy', optimizer=sgd, metrics= ['accuracy']) # Training and saving the model hist= model.fit (np.array (train_x), np.array (train_y), epochs=200, batch_size=5,verbose=1) model.save (' chatbot_model.h6', hist) print ("model is created")
Step 5: interact with chatbots
The model is ready to chat, and now create a good graphical user interface for the chatbot in a new file. You can name the file gui_chatbot.py
In the GUI file, use the Tkinter module to build the structure of the desktop application, then capture the user message, and perform some preprocessing again before entering the message into the training model.
The model then predicts the label of the user message and randomly selects the response from the response list in the intents file.
This is the complete source code for the GUI file.
Importnltk fromnltk.stem importWordNetLemmatizer lemmatizer = WordNetLemmatizer () importpickle importnumpy as np fromkeras.models importload_model model= load_model ('chatbot_model.h6') importjson importrandom intents= json.loads (open (' intents.json'). Read () words= pickle.load (open ('words.pkl','rb')) classes= pickle.load (open (' classes.pkl') 'rb')) defclean_up_sentence (sentence): # tokenize the pattern-splittingwords into array sentence_words = nltk.word_tokenize (sentence) # stemming every word-reducing tobase form sentence_words = [lemmatizer.lemmatize (word.lower ()) for word in sentence_words] returnsentence_words # return bag of words array: 0 or 1for words that exist in sentence defbag_of_words (sentence, words) Show_details=True): # tokenizing patterns sentence_words = clean_up_sentence (sentence) # bag of words-vocabulary matrix bag= [0] * len (words) fors in sentence_words: fori,word inenumerate (words): ifword = = s: # assign 1 if current word is in thevocabulary position bag [I] = 1 ifshow_details: print ("found in bag:%s"% word) return (np.array (bag) defpredict_class (sentence): # filter below thresholdpredictions p = bag_of_words (sentence,words) Show_details=False) res= model.predict (np.array ([p])) [0] ERROR_THRESHOLD = 0.25results= [[iMagner] fori,r inenumerate (res) ifr > ERROR_THRESHOLD] # sorting strength probability results.sort (key=lambdax: X [1], reverse=True) return_list = [] forr in results: return_list.append ({"intent": classes [r [0]], "probability": str (r [1)}) returnreturn_list defgetResponse (ints Intents_json): tag= ints [0] ['intent'] list_of_intents = intents_json [' intents'] fori in list_of_intents: if (I ['tag'] = = tag): result= random.choice (I [' responses']) break returnresult # Creating tkinter GUI importtkinter fromtkinter import * defsend (): msg= EntryBox.get ("1.0", 'end-1c'). Strip () EntryBox.delete ("0", END) ifmsg! =': ChatBox.config (state=NORMAL) ChatBox.insert (END) "You:" + msg+'\ n\ n') ChatBox.config (foreground= "# 446665", font= ("Verdana", 12) ints= predict_class (msg) res= getResponse (ints,intents) ChatBox.insert (END, "Bot:" + res+'\ n\ n') ChatBox.config (state=DISABLED) ChatBox.yview (END) root= Tk () root.title ("Chatbot") root.geometry ("400x500" root.resizable (width=FALSE, height=FALSE) # Create Chat window ChatBox= Text (root, bd=0, bg= "white", height= "8") Width= "50", font= "Arial",) ChatBox.config (state=DISABLED) # Bind scrollbar to Chat window scrollbar= Scrollbar (root,command=ChatBox.yview, cursor= "heart") ChatBox ['yscrollcommand'] = scrollbar.set # Create Button to send message SendButton = Button (root,font= ("Verdana", 12), text= "Send", width= "12", height=5, bd=0,bg= "# f9a602", activebackground= "# 3c9d9b", fg='#000000', command=send) # Create the box to enter message EntryBox= Text (root, bd=0,bg= "white", width= "29") Height= "5", font= "Arial") # EntryBox.bind (", send) # Place all components on the screen scrollbar.place (Xerox 376, height=386) ChatBox.place (Xerox 6, height=386,width=370) EntryBox.place (Xerox 128, height=90,width=265) SendButton.place (Xerox 6, height=90) root.mainloop ()
Run a chat robot
Now there are two separate files, one is train_chatbot.py, which is first used to train the model.
Pythontrain_chatbot.py so far, I believe you have a deeper understanding of "how to use Python to achieve chat robot project". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.