1

在使用亚马逊评论刮板构建此数据框后，我调用 nlp 以标记并创建一个包含已处理评论的新列作为“文档”

但是，现在我正在尝试创建一个模式以分析 doc 列中的评论，但我不断了解匹配项，这让我认为我错过了一个预处理步骤，或者可能没有将匹配器指向正确的方向。

虽然以下代码执行时没有任何错误，但我收到一个带有 0 的匹配列表 - 即使我知道该单词存在于 doc 列中。spaCy 的文档仍然有点薄，我不太确定 matcher.add 是否正确，因为教程中的具体内容

matcher.add("Name_of_List", None, pattern)

返回一个错误，说明这个类只需要 2 个参数。

问题：我需要更改哪些内容才能准确分析创建的模式的 df doc 列？

谢谢！

完整代码：

import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns

import spacy
from spacy.matcher import Matcher

nlp = spacy.load('en_core_web_md')


df = pd.read_csv('paper_towel_US.csv')

#calling on NLP to return processed doc for each review
df['doc'] = [nlp(body) for body in df.body]


# Sum the number of tokens in each Doc
df['num_tokens'] = [len(token) for token in df.doc]



#calling matcher to create pattern
matcher = Matcher(nlp.vocab)


pattern =[{"LEMMA": "love"},
          {"OP":"+"}
          
          ]
matcher.add("QUALITY_PATTERN", [pattern])


def find_matches(doc):
    spans = [doc[start:end] for _, start, end in matcher(doc)]
    for span in spacy.util.filter_spans(spans):
        return ((span.start, span.end, span.text))
    
    
df['doc'].apply(find_matches)

df 样本通过复制df.iloc[596:600, :].to_clipboard(sep=',')

,product,title,rating,body,doc,num_tokens
596,Amazon.com: Customer reviews: Bamboo Towels - Heavy Duty Machine Washable Reusable Rayon Towels - One roll replaces 6 months of towels! 1 Pack,Awesome!,5,Great towels!,Great towels!,3
597,Amazon.com: Customer reviews: Bamboo Towels - Heavy Duty Machine Washable Reusable Rayon Towels - One roll replaces 6 months of towels! 1 Pack,Good buy!,5,Love these,Love these,2
598,Amazon.com: Customer reviews: Bamboo Towels - Heavy Duty Machine Washable Reusable Rayon Towels - One roll replaces 6 months of towels! 1 Pack,Meh,3,"Does not clean countertop messes well. Towels leave a large residue. They are durable, though","Does not clean countertop messes well. Towels leave a large residue. They are durable, though",18
599,Amazon.com: Customer reviews: Bamboo Towels - Heavy Duty Machine Washable Reusable Rayon Towels - One roll replaces 6 months of towels! 1 Pack,Exactly as Described. Packaged Well and Mailed Promptly,4,Exactly as Described. Packaged Well and Mailed Promptly,Exactly as Described. Packaged Well and Mailed Promptly,9
4

1 回答 1

1

您正在尝试从"df.doc"字符串中获取匹配项doc = nlp("df.doc")。您需要从df['doc']列中提取匹配项。

一个示例解决方案是删除doc = nlp("df.doc")并使用nlp = spacy.load('en_core_web_sm')：

def find_matches(doc):
    spans = [doc[start:end] for _, start, end in matcher(doc)]
    for span in spacy.util.filter_spans(spans):
        return ((span.start, span.end, span.text))

>>> df['doc'].apply(find_matches)
0                  None
1    (0, 2, Love these)
2                  None
3                  None
Name: doc, dtype: object

完整的代码片段：

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

import spacy
from spacy.matcher import Matcher

nlp = spacy.load('en_core_web_sm')


df = pd.read_csv(r'C:\Users\admin\Desktop\s.txt')

#calling on NLP to return processed doc for each review
df['doc'] = [nlp(body) for body in df.body]


# Sum the number of tokens in each Doc
df['num_tokens'] = [len(token) for token in df.doc]

#calling matcher to create pattern
matcher = Matcher(nlp.vocab)


pattern =[{"LEMMA": "love"},
          {"OP":"+"}
          
          ]
matcher.add("QUALITY_PATTERN", [pattern])

#doc = nlp("df.doc")

#matches = matcher(doc)
def find_matches(doc):
    spans = [doc[start:end] for _, start, end in matcher(doc)]
    for span in spacy.util.filter_spans(spans):
        return ((span.start, span.end, span.text))

print(df['doc'].apply(find_matches))
于 2021-07-03T12:09:16.053 回答