python - 将 Vader 情绪分析写入 csv 中的新列

Question

我有一个旅行顾问评论的 csv 文件。有四列：

人、标题、评级、评论、review_date。

我希望此代码执行以下操作：

1. 在 csv 中，创建一个名为“tarate”的新列。
2. 用“pos”、“neg”或“neut”填充“tarate”。它应该读取“评级”中的数值。如果“评级”>=40，则“tarate”== 'pos'；“tarate” == 'neut' 如果“rating” == 30；“tarate” == 'neg' 如果“rating”<30。
3. 接下来，通过 SentimentIntensityAnalyzer 运行“review”列。
4. 将输出记录在名为“scores”的新 csv 列中
5. 使用“pos”和“neg”分类为“复合”值创建一个单独的 csv 列
6. 运行 sklearn.metrics 工具将旅行顾问评级（“tarate”）与“compound”进行比较。这可以打印。

部分代码基于 [http://akashsenta.com/blog/sentiment-analysis-with-vader-with-python/]

这是我的 csv 文件：[https://github.com/nsusmann/vadersentiment]

我遇到了一些错误。我是一个初学者，我想我被诸如指向特定列和 lambda 函数之类的东西绊倒了。

这是代码：

# open command prompt
# import nltk
# nltk.download()
# pip3 install pandas
# pip3 installs sci-kitlearn
# pip3 install matplotlib
# pip3 install seaborn
# pip3 install vaderSentiment
#pip3 install openpyxl

import pandas
import nltk
nltk.download([
    "vader_lexicon",
    "stopwords"])
from nltk.sentiment import SentimentAnalyzer
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from nltk.sentiment.util import *
from nltk import tokenize
from nltk.corpus import stopwords
import pandas as pd
import numpy as np
from collections import Counter
import re
import math
import html
import sklearn
import sklearn.metrics as metrics
from sklearn.metrics import mutual_info_score
from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer, ENGLISH_STOP_WORDS
import matplotlib.pyplot as plt
import seaborn as sns
from pprint import pprint
import openpyxl

# open the file to save the review
import csv
outputfile = open('D:\Documents\Archaeology\Projects\Patmos\TextAnalysis\Sentiment\scraped_cln_sent.csv', 'w', newline='')
df = csv.writer(outputfile)

#open Vader Sentiment Analyzer 
from nltk.sentiment.vader import SentimentIntensityAnalyzer

#make SIA into an object
analyzer = SentimentIntensityAnalyzer()

#create a new column called "tarate"
df['tarate'],
#populate column "tarate". write pos, neut, or neg per row based on column "rating" value
df.loc[df['rating'] >= 40, ['tarate'] == 'Pos',
df.loc[df['rating'] == 30, ['tarate'] == 'Neut',
df.loc[df['rating'] <= 20, ['tarate'] == 'Neg', 

#use polarity_scores() to get sentiment metrics and write them to new column "scores"
df.head['scores'] == df['review'].apply(lambda review: sid.polarity_scores['review'])

#extract the compound value of the polarity scores and write to new column "compound"
df['compound'] = df['scores'].apply(lambda d:d['compound'])

#using column "compound", determine whether the score is <0> and write new column "score" recording positive or negative
df['score'] = df['compound'].apply(lambda score: 'pos' if score >=0 else 'neg')
ta.file()
                                           
#get accuracy metrics. this will compare the trip advisor rating (text version recorded in column "tarate") to the sentiment analysis results in column "score"
from sklearn.metrics import accuracy_score,classification_report,confusion_matrix                                           
accuracy_score(df['tarate'],df['score'])

print(classification_report(df['tarate'],df['score']))     ```

Answer 1

您无需在填充之前创建新列。此外，您在行尾有虚假逗号。不要那样做; 逗号和 Python 中表达式的结尾将其转换为元组。还要记住这=是赋值运算符，并且==是比较。

pandas“loc”函数采用行索引器和列索引器：

#populate column "tarate". write pos, neut, or neg per row based on column "rating" value
df.loc[df['rating'] >= 40, 'tarate'] = 'Pos'
df.loc[df['rating'] == 30, 'tarate'] = 'Neut'
df.loc[df['rating'] <= 20, 'tarate'] = 'Net'

请注意，NaN对于介于 20 和 30 之间的值以及介于 30 和 40 之间的值，这将在列中留下（不是数字）。

我不知道你想在这里做什么，但这是不对的：

#extract the compound value of the polarity scores and write to new column "compound"
df['compound'] = df['scores'].apply(lambda d:d['compound'])

df['scores']不会包含名为“compound”的列，这是您在 lambda 中要求的。

Answer 2

我建议查找列表推导、谷歌“pandas 应用方法”和“pandas lambda 示例”以更熟悉它们。

对于一些示例代码：

import pandas as pd

#create a demo dataframe called 'df'
df = pd.DataFrame({'rating': [12, 42, 40, 30, 31, 56, 8, 88, 39, 79]})

这为您提供了一个看起来像这样的数据框（只有一个名为“rating”的列，其中包含整数）：

   rating
0      12
1      42
2      40
3      30
4      31
5      56
6       8
7      88
8      39
9      79

使用该列根据其中的值制作另一个列可以像这样完成......

#create a new column called 'tarate' and using a list comprehension
#assign a string value of either 'pos', 'neut', or 'neg' based on the 
#numeric value in the 'rating' column (it does this going row by row)
df['tarate'] = ['pos' if x >= 40 else 'neut' if x == 30 else 'neg' for x in df['rating']]

#output the df
print(df)

输出：

   rating tarate
0      12    neg
1      42    pos
2      40    pos
3      30   neut
4      31    neg
5      56    pos
6       8    neg
7      88    pos
8      39    neg
9      79    pos