1

我是文本分析的新手，目前正在尝试 R 中的#Quanteda 包以满足我的需要。我想为某些特定分配不同的数字权重并测试模型的准确性。我尝试了此处其他线程中提到的方法，方法是保留 dfm 类， 将权重分配给 R 中的不同特征，但无法获得正确的输出。任何帮助，将不胜感激。

这是我尝试过的

##install.packages("quanteda")
require(quanteda)
str <- c("apple is better than banana", "banana banana apple much  
better","much much better new banana")

weights <- c(apple = 5, banana = 3, much = 0.5)
myDfm <- dfm(str, remove = stopwords("english"), verbose = FALSE)

#output
##Document-feature matrix of: 3 documents, 5 features.
##3 x 5 sparse Matrix of class "dfmSparse"
##   features
##docs    apple better banana much new
##text1     1      1      1    0   0
##text2     1      1      2    1   0
##text3     0      1      1    2   1

newweights <- weights[featnames(myDfm)]
# reassign 1 to non-matched NAs
newweights[is.na(newweights)] <- 1

# this does not works for me - see the output
myDfm * newweights

##output
##Document-feature matrix of: 3 documents, 5 features.
##3 x 5 sparse Matrix of class "dfmSparse"
##   features
##docs    apple better banana much new
##text1     5    0.5    1.0    0   0
##text2     1    1.0    6.0    5   0
##text3     0    5.0    0.5    2   1

环境细节

platform x86_64-w64-mingw32
arch x86_64
os mingw32
system x86_64, mingw32
status
major 3
minor 2.2
year 2015
month 08
day 14
svn rev 69053
language R
version.string R version 3.2.2 (2015-08-14) 昵称 Fire Safety

4

1 回答 1

0

这显然与dfm 类所基于的Matrix*包中的运算符有关。这有效：

> matrix(1:6, nrow = 3) * c(2, 3)
     [,1] [,2]
[1,]    2   12
[2,]    6   10
[3,]    6   18

但这不会：

> Matrix::Matrix(matrix(1:6, nrow = 3)) * c(2, 3)
Error in Matrix(matrix(1:6, nrow = 3)) * c(2, 3) : 
  length of 2nd arg does not match dimension of first

在我们解决这个问题之前，这里有一个解决方法：使权重向量逐个元素地对应于 dfm。

myDfm * rep(newweights, each = ndoc(myDfm))
## Document-feature matrix of: 3 documents, 5 features.
## 3 x 5 sparse Matrix of class "dfmSparse"
##        features
## docs    apple better banana much new
##   text1     5      1      3  0     0
##   text2     5      1      6  0.5   0
##   text3     0      1      3  1.0   1

更新：

这不是一个错误，而是一个特性，并且与向量如何newweights被回收以符合与其相乘的矩阵有关。R 使用列优先顺序回收此向量，因此它正在创建以下矩阵，该矩阵在此示例中有效（尽管不是您想要的），以执行逐个元素的乘法：

matrix(rep(newweights, 3), nrow = 3)
##      [,1] [,2] [,3] [,4] [,5]
## [1,]    5  0.5  1.0    1  3.0
## [2,]    1  1.0  3.0    5  0.5
## [3,]    3  5.0  0.5    1  1.0

如果您想使用原始策略，这将起作用：

t(t(myDfm) * newweights)
## Document-feature matrix of: 3 documents, 5 features (26.7% sparse).
## 3 x 5 sparse Matrix of class "dfmSparse"
##        features
## docs    apple better banana much new
##   text1     5      1      3  0     0
##   text2     5      1      6  0.5   0
##   text3     0      1      3  1.0   1

因为回收现在发生在功能上，而不是文档上。

于 2016-10-06T06:04:50.417 回答