Python and Sentiment Analysis

  • 1/10/2016
  • 4-minute read

While looking for datasets to throw at sklearn , I came across UCI Sentiment Labelled Sentences Data Set.

UCI is providing us with positive / negative tagging on real world data, the data comes from three sources ( Amazon , Yelp , and IMDB ).

The only problem is the format is a little strange.. We have a .txt file for each source, this is a raw unstructured  formatting, plus not every line is tagged with sentiment.

To make the data easier to interact with, I generated a json file with only the results containing sentiment. Go ahead and download it.

$ zcat sentiment.json.gz | head -n 25
[
 {
 "result": 0,
 "source": "amazon_cells_labelled.txt",
 "label": "negative",
 "text": "So there is no way for me to plug it in here in the US unless I go by a converter."
 },
 {
 "result": 1,
 "source": "amazon_cells_labelled.txt",
 "label": "positive",
 "text": "Good case, Excellent value."
 },
 {
 "result": 1,
 "source": "amazon_cells_labelled.txt",
 "label": "positive",
 "text": "Great for the jawbone."
 },
 {
 "result": 0,
 "source": "amazon_cells_labelled.txt",
 "label": "negative",
 "text": "Tied to charger for conversations lasting more than 45 minutes.MAJOR PROBLEMS!!"
 },

Lets jump into an IPython interrupter and load the data:

In [1]: import json

In [2]: raw = open('sentiment.json').read()

Now that we have the data as a Python dictionary, create a DataFrame in the proper format:

In [1]: data = get_data_frame('sentiment.json')

In [2]: data.shape
Out[2]: (3000, 2)

Next lets split our full dataset into a training, and testing dataset:

In [1]: train, test = get_train_test_data(data, size=0.2)

In [2]: train.shape
Out[2]: (2400, 2)

In [3]: test.shape
Out[3]: (600, 2)

We are now set to run a bit of accuracy testing:

In [1]: test_predict(train, test)
Out[1]: {
  'test_score': 0.80000000000000004,
  'train_score': 0.98375000000000001
}

We can slice our full dataset a few more times, just to make sure our accuracy test is.. accurate:

In [1]: train, test = get_train_test_data(data, size=0.2)

In [2]: test_predict(train, test)
Out[2]: {
  'test_score': 0.79000000000000004,
  'train_score': 0.98416666666666663
}

In [3]: train, test = get_train_test_data(data, size=0.2)

In [4]: test_predict(train, test)
Out[4]: {
  'test_score': 0.80666666666666664,
  'train_score': 0.98291666666666666
}

In [5]: train, test = get_train_test_data(data, size=0.5)

In [6]: test_predict(train, test)
Out[6]: {
  'test_score': 0.79466666666666663,
  'train_score': 0.98999999999999999
}

All that is left is to feed in the entire dataset and predict on new sentences:

In [1]: predict(data, 'This was the worst experience.')
Out[1]: [
  (u'positive', 0.17704535094140364),
  (u'negative', 0.82295464905859583)
]

In [2]: predict(data, 'The staff here was fabulous')
Out[2]: [
  (u'negative', 0.20651083543376234),
  (u'positive', 0.79348916456623764)
]

In [1]: predict(data, 'I hate you')
Out[1]: [
  (u'positive', 0.22509671479185445),
  (u'negative', 0.77490328520814555)
]

In [2]: predict(data, 'I love you')
Out[2]: [
  (u'negative', 0.10593166714256422),
  (u'positive', 0.89406833285743614)
]

Lets put it all together by looking at the Python functions:

import re
import json
import random
import string

from pandas import DataFrame, concat

from sklearn.pipeline import Pipeline
from sklearn.naive_bayes import MultinomialNB
from sklearn.neighbors import NearestNeighbors
from sklearn.cross_validation import train_test_split
from sklearn.feature_extraction.text import TfidfTransformer, \
 CountVectorizer


# update the pipeline to get best test results!
steps = [
 ('count_vectorizer', CountVectorizer(
 stop_words='english', ngram_range=(1, 2))),
 ('tfidf_transformer', TfidfTransformer()),
 ('classifier', MultinomialNB())
]


def id_generator(size=6):
 """
 Return random string.
 """
 chars = string.ascii_uppercase + string.digits
 return ''.join(random.choice(chars) for _ in range(size))

def get_data_frame(filename):
 """
 Read tweets.json from directory and return DataFrame.
 """

 raw = dict(ids=[], text=[])

 # open file and read as json
 _data = open(filename)
 data = json.loads(_data.read())

 # loop over all tweets in json file
 for d in data:

 # update raw list with tweet values.
 raw['text'].append(
 dict(text=d['text'], classification=d['label']))
 raw['ids'].append(
 id_generator(size=12))

 return DataFrame(raw['text'], index=raw['ids'])

def merge(*args):
 """
 Merge two or more DataFrames.
 """
 return concat(args)

def get_train_test_data(data, size=0.2):
 """
 Split DataFrame and return a training and testing set.
 """
 train, test = train_test_split(data, test_size=size)
 return train, test

def test_predict(train, test):
 """
 Run predictions on training and test data,
 then return scores.
 """

 pipeline = Pipeline(steps)
 pipeline.fit(train.text.values, train.classification.values)

 train_score = pipeline.score(
 train.text.values, train.classification.values)
 test_score = pipeline.score(
 test.text.values, test.classification.values)

 return dict(train_score=train_score, test_score=test_score)

def predict(data, text):

 pipeline = Pipeline(steps)
 pipeline.fit(data.text.values, data.classification.values)

 res = zip(pipeline.classes_, pipeline.predict_proba([text])[0])
 return sorted(res, key=lambda x:x[1])
DevelopmentPythonMachine Learning