Shulou(Shulou.com)06/02 Report--

This article shows you how to use Python to build a scalable social media emotional analysis service, which is concise and easy to understand. I hope you can learn something from the details of this article.

Explore Python library spaCy and vaderSentiment prerequisites

A terminal shell

Python language binaries in shell (version 3.4 +)

Pip commands for installing Python packages

(optional) A Python virtual environment isolates your work from the system

Configure the environment

Before you start writing code, you need to install the spaCy and vaderSentiment packages to set up the Python environment, and download a language model to help you analyze. Fortunately, most of the operations are easily done on the command line.

In shell, enter the following command to install the spaCy and vaderSentiment packages:

Pip install spacy vaderSentiment

After the command installation is complete, install the language model that spaCy can use for text analysis. The following command downloads and installs the English model using the spaCy module:

Python-m spacy download en_core_web_sm

Once these libraries and models are installed, you can start coding.

A simple text analysis

Use the Python interpreter interaction mode to write some code to analyze a single piece of text. Start the Python environment first:

$pythonPython 3.6.8 (default, Jan 31 2019, 09:38:34) [GCC 8.2.1 20181215 (Red Hat 8.2.1-6)] on linuxType "help", "copyright", "credits" or "license" for more information. > > (your Python interpreter version may print differently. )

1. Import the required modules:

> > import spacy > from vaderSentiment import vaderSentiment

2. Load the English language model from spaCy:

> > english = spacy.load ("en_core_web_sm")

3. Process a piece of text. This example shows a very simple sentence that we hope will bring us some positive emotions:

> result = english ("I like to eat applesauce with sugar and cinnamon.")

4. Collect sentences from the processed results. SpaCy has identified and processed the entities in the phrase, which generates emotion for each sentence (even if there is only one sentence in this case):

> sentences = [str (s) for s in result.sents]

5. Create a parser using vaderSentiments:

> > analyzer = vaderSentiment.SentimentIntensityAnalyzer ()

6. Emotional analysis of sentences:

> sentiment = [analyzer.polarity_scores (str (s)) for s in sentences]

The sentiment variable now contains the polarity score of the example sentence. Print out this value and see how it analyzes the sentence.

> print (sentiment) [{'neg': 0.0,' neu': 0.737, 'pos': 0.263,' compound': 0.3612}]

What does this structure mean?

Ostensibly, this is an array with only one dictionary object. If there are multiple sentences, each sentence corresponds to a dictionary object. There are four keys in the dictionary that correspond to different types of emotions. The neg key represents negative emotions because no negative emotions are reported in this example, as evidenced by the 0. 0 value. The neu key, which represents neutral emotion, has a high score of 0.737. The pos key represents positive emotion, with a moderate score of 0.263. * the cmpound key represents the overall score of the text, which can range from negative to positive, and 0.3612 indicates a little more emotion on the positive side.

To see how these values may change, you can do a small experiment with the code you have entered. The following code block shows how to evaluate the emotional score of a similar sentence.

> result = english ("I love applesauce!") > sentences = [str (s) for s in result.sents] > sentiment = [analyzer.polarity_scores (str (s) for s in sentences] > print (sentiment) [{'neg': 0.0,' neu': 0.182, 'pos': 0.818,' compound': 0.6696}]

You can see that by changing the example sentence into a very positive sentence, the value of sentiment has changed dramatically.

Establish an emotional analysis service

Now that you've assembled the basic code blocks for emotional analysis, let's turn these things into a simple service.

In this demonstration, you will create a RESTful HTTP server using the Python Flask package. This service will accept English text data and return the results of emotion analysis. Note that this sample service is used to learn the technology involved, not to put it into production.

prerequisite

A terminal shell

Python language binaries in shell (version 3.4 +)

Install the pip command for the Python package

Curl command

A text editor

(optional) A Python virtual environment isolates your work from the system

Configure the environment

This environment is almost the same as in the previous section, except that the Flask package is added to the Python environment.

1. Dependencies required for installation:

Pip install spacy vaderSentiment flask

2. Install the English language model of spaCy:

Python-m spacy download en_core_web_sm creates application files

Open the editor and create a file called app.py. Add the following (don't worry, we'll explain each line):

Import flaskimport spacyimport vaderSentiment.vaderSentiment as vader app = flask.Flask (_ name__) analyzer = vader.SentimentIntensityAnalyzer () english = spacy.load ("en_core_web_sm") def get_sentiments (text): result = english (text) sentences = [str (sent) for sent in result.sents] sentiments = [analyzer.polarity_scores (str (s) for s in sentences] return sentiments @ app.route ("/", methods= ["POST") "GET"]) def index (): if flask.request.method = "GET": return "To access this service send a POST request to this URL with"\ "the text you want analyzed in the body." Body = flask.request.data.decode ("utf-8") sentiments = get_sentiments (body) return flask.json.dumps (sentiments)

Although this source file is not very large, it is very dense. Let's look at the various parts of the application and explain what they are doing.

Import flaskimport spacyimport vaderSentiment.vaderSentiment as vader

The first three lines introduce the packages needed to perform language analysis and the HTTP framework.

App = flask.Flask (_ _ name__) analyzer = vader.SentimentIntensityAnalyzer () english = spacy.load ("en_core_web_sm")

The next three lines of code create some global variables. * variable app, which is the main entry point that Flask uses to create HTTP routes. The second variable, analyzer, which is of the same type used in the previous example, will be used to generate emotional scores. * A variable english, which is also of the same type used in the previous example, will be used to comment and mark up initial text input.

You may wonder why these variables are declared globally. For app variables, this is a standard procedure for many Flask applications. However, for analyzer and english variables, the decision to set them as global variables is based on the load time associated with the class involved. Although the load time may seem short, these delays can have a negative impact on performance when it runs in the context of the HTTP server.

Def get_sentiments (text): result = english (text) sentences = [str (sent) for sent in result.sents] sentiments = [analyzer.polarity_scores (str (s)) for s in sentences] return sentiments

This part is the core of the service-a function used to generate emotional values from a string of text. You can see that the operation in this function corresponds to the command you previously ran in the Python interpreter. Here they are encapsulated in a function definition, where the text source is passed in as a text variable and the * * sentiments variable is returned to the caller.

App.route ("/", methods= ["POST", "GET"]) def index (): if flask.request.method = = "GET": return "To access this service send a POST request to this URL with"\ "the text you want analyzed in the body." Body = flask.request.data.decode ("utf-8") sentiments = get_sentiments (body) return flask.json.dumps (sentiments)

A function of the source file contains the logic that instructs Flask to configure the HTTP server for the service. It starts with a line that associates HTTP routing / with the request methods POST and GET.

After the function defines the line, the if clause detects whether the request method is GET. If the user sends this request to the service, the following line returns a text message indicating how to access the server. This is mainly for the convenience of end users.

The next line uses the flask.request object to get the body of the request, which should contain the text string to be processed. The decode function converts a byte array into an available formatted string. The decoded text message is passed to the get_sentiments function to generate an emotional score. * the score is returned to the user through the HTTP framework.

You should save the file now and return shell if you haven't already saved it.

Run emotional service

When everything is in place, it is easy to run the service using Flask's built-in debugging server. To start the service, enter the following command from the same directory as the source file:

FLASK_APP=app.py flask run

Now you will see some output from the server in shell, and the server will be running. To test whether the server is running, you need to open the second shell and use the curl command.

First, enter the following command to check if the instruction information is printed:

Curl http://localhost:5000

You should see the explanatory message:

To access this service send a POST request to this URI with the text you want analyzed in the body.

Next, run the following command to send a test message to view the emotional analysis:

Curl http://localhost:5000-header "Content-Type: application/json"-data "I love applesauce!"

The response you get from the server should be similar to the following:

[{"compound": 0.6696, "neg": 0.6696, "neu": 0.182, "pos": 0.818}]

Congratulations! You have now implemented a RESTful HTTP emotion analysis service. You can find the reference implementation of this service and all the code in this article on GitHub.

Continue to explore

Now that you understand the principles and mechanisms behind natural language processing and emotional analysis, here are some ways to further explore the topic.

Create a streaming emotion analyzer on OpenShift

While it is convenient to create a local application to study emotional analysis, you need to be able to deploy the application for a wider range of uses. By following the instructions and code provided by Radnaalytics.io, you will learn how to create an emotion analyzer that can be containerized and deployed to the Kubernetes platform. You will also learn how to use Apache Kafka as a framework for event-driven messaging and how Apache Spark can be used as a distributed computing platform for emotion analysis.

Using Twitter API to discover real-time data

Although Radanalytics.io Labs can generate synthetic tweet streams, you can not be limited to synthetic data. In fact, anyone with a Twitter account can use the Tweepy Python package to access Twitter streaming API for emotional analysis of tweets.

The above is how to use Python to build scalable social media emotion analysis services. Have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.