0

我有一个数据集，其中每一行都包含一串这种类型的文本

1)list(text = \"incredible hosts\", relevance = 0.87518, count = 1), list(text = \"Japan\", relevance = 0.675236, count = 1), list(text = \"support\", relevance = 0.625663, count = 1), list(text = \"result\", relevance = 0.359757, count = 1)


2)list(text = \"British fleet\", relevance = 0.912888, count = 1), list(text = \"worst maritime disasters\", relevance = 0.904047, count = 1), list(text = \"British history\", relevance = 0.755491, count = 1), list(text = \"Scilly Isles\", relevance = 0.716508, count = 1), list(text = \"sailors\", relevance = 0.691141, count = 1), list(text = \"evening\", relevance = 0.597375, count = 1), list(text = \"Tragedy\", relevance = 0.577141, count = 1), list(text = \"prize\", relevance = 0.565035, count = 1), list(text = \"rocks\", relevance = 0.543257, count = 1), list(text = \"innovation\", relevance = 0.529463, count = 1), list(text = \"longitude\", relevance = 0.335207, count = 1)

基本上我想只提取 \" 和 \" 之间包含的文本字符串

并获得这样的东西

1) "incredible hosts, Japan, support , result"
2) "British fleet, worst maritime disasters, British history, scilly Isles, sailors, evening, etc..."

此外，我想创建一个数据框，帮助跟踪每段文本的文本中包含的相关性分数（考虑到不同的原始数据可能具有不同数量的文本），以便获得如下信息：

 col1                 col2.   col3.    col4.   col5.     col6.....  colA1    colA2.  .....
 incredible hosts     Japon  support  result    NA.      NA        0.87518.   0.675236....
 british fleet.       worst marit.......

基本上等于一行中文本的最大数量的列数，与分数对应的列相同（每个相关性分数指的是一段文本，因此它们是相同的数字）。

如果我能找到一种方法来首先提取文本片段并用逗号分隔它们，然后对相关性分数做同样的事情，我认为我可以轻松地将两者合并到一个数据框中。所以问题主要是从该文本中提取这两件事。

预先感谢您的帮助，

卡罗

4

2 回答 2

2

您在上面显示的字符串几乎是正确的R代码。因此，只需最少的修改，我们就可以直接将数据读入R：

txt1 <- 'list(text = \"incredible hosts\", relevance = 0.87518, count = 1), list(text = \"Japan\", relevance = 0.675236, count = 1), list(text = \"support\", relevance = 0.625663, count = 1), list(text = \"result\", relevance = 0.359757, count = 1)'
txt2 <- 'list(text = \"British fleet\", relevance = 0.912888, count = 1), list(text = \"worst maritime disasters\", relevance = 0.904047, count = 1), list(text = \"British history\", relevance = 0.755491, count = 1), list(text = \"Scilly Isles\", relevance = 0.716508, count = 1), list(text = \"sailors\", relevance = 0.691141, count = 1), list(text = \"evening\", relevance = 0.597375, count = 1), list(text = \"Tragedy\", relevance = 0.577141, count = 1), list(text = \"prize\", relevance = 0.565035, count = 1), list(text = \"rocks\", relevance = 0.543257, count = 1), list(text = \"innovation\", relevance = 0.529463, count = 1), list(text = \"longitude\", relevance = 0.335207, count = 1)'

txt1 <- gsub("text = ", "id = 1, text = ", txt1) # this is just if you want to have an ID later on
txt2 <- gsub("text = ", "id = 2, text = ", txt2)

list1 <- eval(parse(text = paste0("list(", txt1, ")")))
list2 <- eval(parse(text = paste0("list(", txt2, ")")))


df <- dplyr::bind_rows(list1, list2)
df
#> # A tibble: 15 x 4
#>       id text                     relevance count
#>    <dbl> <chr>                        <dbl> <dbl>
#>  1     1 incredible hosts             0.875     1
#>  2     1 Japan                        0.675     1
#>  3     1 support                      0.626     1
#>  4     1 result                       0.360     1
#>  5     2 British fleet                0.913     1
#>  6     2 worst maritime disasters     0.904     1
#>  7     2 British history              0.755     1
#>  8     2 Scilly Isles                 0.717     1
#>  9     2 sailors                      0.691     1
#> 10     2 evening                      0.597     1
#> 11     2 Tragedy                      0.577     1
#> 12     2 prize                        0.565     1
#> 13     2 rocks                        0.543     1
#> 14     2 innovation                   0.529     1
#> 15     2 longitude                    0.335     1

从这里可以更容易地将它们整理成您可能想要的任何形式。

更新

在您发表评论后，我更改了答案以展示如何将这种方法与更大的数据集一起使用并将其放入 quanteda：

假设您读取了数据，并且每个文本现在都是向量中的值：

txt <- c('list(text = \"incredible hosts\", relevance = 0.87518, count = 1), list(text = \"Japan\", relevance = 0.675236, count = 1), list(text = \"support\", relevance = 0.625663, count = 1), list(text = \"result\", relevance = 0.359757, count = 1)',
         'list(text = \"British fleet\", relevance = 0.912888, count = 1), list(text = \"worst maritime disasters\", relevance = 0.904047, count = 1), list(text = \"British history\", relevance = 0.755491, count = 1), list(text = \"Scilly Isles\", relevance = 0.716508, count = 1), list(text = \"sailors\", relevance = 0.691141, count = 1), list(text = \"evening\", relevance = 0.597375, count = 1), list(text = \"Tragedy\", relevance = 0.577141, count = 1), list(text = \"prize\", relevance = 0.565035, count = 1), list(text = \"rocks\", relevance = 0.543257, count = 1), list(text = \"innovation\", relevance = 0.529463, count = 1), list(text = \"longitude\", relevance = 0.335207, count = 1)')

您可以循环遍历每个元素，而不是更改每个对象：

txt <- lapply(seq_along(txt), function(i) { # this is just if you want to have an ID later on
  gsub("text = ", paste0("id = ", i, ", text = "), txt[i])
})

list <- lapply(txt, function(x) {
  dplyr::bind_rows(eval(parse(text = paste0("list(", x, ")"))))
})

df <- dplyr::bind_rows(list)

一旦你有了data.frame，在你可以在 quanteda 中工作之前，只剩下一点点数据整理工作要做：

library(dplyr)
df_wide <- df %>% 
  group_by(id) %>% 
  summarise(text = paste(text, collapse = " "), relevance = list(relevance))

library(quanteda)

corp <- corpus(df_wide, docid_field = "id", text_field = "text")
corp
#> Corpus consisting of 2 documents and 1 docvar.
corp$documents$relevance
#> [[1]]
#> [1] 0.875180 0.675236 0.625663 0.359757
#> 
#> [[2]]
#>  [1] 0.912888 0.904047 0.755491 0.716508 0.691141 0.597375 0.577141
#>  [8] 0.565035 0.543257 0.529463 0.335207
于 2019-11-21T18:27:42.503 回答
1

这是一个基本的 R 方法，它至少输出所有匹配的引号术语：

x <- "list(text = \"incredible hosts\", relevance = 0.87518, count = 1), list(text = \"Japan\", relevance = 0.675236, count = 1), list(text = \"support\", relevance = 0.625663, count = 1), list(text = \"result\", relevance = 0.359757, count = 1)"
m <- gregexpr("\"(.*?)\"", x)
regmatches(x, m)[[1]]

[1] "\"incredible hosts\"" "\"Japan\""            "\"support\""
[4] "\"result\""
于 2019-11-21T18:22:23.767 回答