0

在我的工作中，我经常使用调查数据，这些数据来自格式非常糟糕的 Excel 文件，这些文件旨在提高可读性，而不是用于任何数据分析。我正在寻找一种方法来清理 R 中的数据并将其转换为变量和观察值的数据框格式。

我知道在 R 中有很多关于数据清理的教程，但根据我的经验，它们主要处理已经是机器可读格式的数据，所以任何帮助都将不胜感激！

这是具有这种形状的原始调查的虚拟示例：

Are you male or female?

           Variable1 Variable2 Variable3 Variable4
Male       n%        n%        n%        n%
Female     n%        n%        n%        n%


How old are you?

           Variable1 Variable2 Variable3 Variable4
18-34      n%        n%        n%        n%
35+        n%        n%        n%        n%

依此类推，空白区域为空单元格/行，每个调查问题的整体都在 A 列中，对应的数据表上方几行，所有问题/数据表都在一个工作表上。

有没有办法用 R 代码转换成这个？

Question                Response Variable1 Variable2 Variable3 Variable4
Are you male or female? Male     n%        n%        n%        n%
Are you male or female? Female   n%        n%        n%        n%
How old are you?        18-34    n%        n%        n%        n%
How old are you?        35+      n%        n%        n%        n%

目前，我正在使用一些 VBA 代码在 excel 中执行此操作，然后读入 R 以进行进一步的分析/可视化，但如果能够跳过 excel 阶段并直接进入 R，那就太好了。

谢谢！

4

1 回答 1

1

这是处理错误整理数据的粗略方法。我制作了一个 csv 格式的文件并将其托管在一个杂项 repo 上：

file <- "https://raw.githubusercontent.com/minerva79/woodpecker/master/data/example.csv"
survey <- readLines(file)

（1）剥离所有白线：

white.lines <- nchar(gsub(",", "", survey))==0
survey <- survey[!white.lines]

[1] "Are you male or female?,,,,"              ",Variable1,Variable2,Variable3,Variable4" "Male,0.5,0.6,0.7,0.8"                    

[4] "Female,0.5,0.4,0.3,0.2"                   "How old are you?,,,,"                     ",Variable1,Variable2,Variable3,Variable4"
[7] "18-34,0.4,0.5,0.7,0.1"                    "35+,0.6,0.5,0.3,0.9"

(2) 识别头部位置

headers <- substring(survey, 1,1) == ","
survey[headers]

[1] ",Variable1,Variable2,Variable3,Variable4" ",Variable1,Variable2,Variable3,Variable4"

(3)根据headers位置找到问题位置

header_pos <- (1:length(survey))[headers]
qn_pos <- header_pos - 1 

qn <- survey[qn_pos] %>% gsub(",", "", .)
qn

[1] "Are you male or female?" "How old are you?"

(4) 确定表格的行（从header_pos到qn_pos-1或length(survey)：

qn_pos <- c(qn_pos - 1, length(survey))
tabs <- lapply(1:length(qn), function(x)survey[header_pos[x]:qn_pos[x+1]])
tabs

[[1]]
[1] ",Variable1,Variable2,Variable3,Variable4" "Male,0.5,0.6,0.7,0.8"                     "Female,0.5,0.4,0.3,0.2"                  

[[2]]
[1] ",Variable1,Variable2,Variable3,Variable4" "18-34,0.4,0.5,0.7,0.1"                    "35+,0.6,0.5,0.3,0.9"

(5) 将每个列表对象读取为表格：

tabs <- lapply(tabs, function(x)read.table(text=x, sep=",", header=T, row.names=1))
tabs

[[1]]
       Variable1 Variable2 Variable3 Variable4
Male         0.5       0.6       0.7       0.8
Female       0.5       0.4       0.3       0.2

[[2]]
      Variable1 Variable2 Variable3 Variable4
18-34       0.4       0.5       0.7       0.1
35+         0.6       0.5       0.3       0.9

(6) mutate question and response，和rbind：

tabs <- lapply(1:length(tabs), function(x) tabs[[x]] %>% mutate(Question= qn[x], Response=row.names(.)))
do.call(rbind, tabs)

  Variable1 Variable2 Variable3 Variable4                Question Response
1       0.5       0.6       0.7       0.8 Are you male or female?     Male
2       0.5       0.4       0.3       0.2 Are you male or female?   Female
3       0.4       0.5       0.7       0.1        How old are you?    18-34
4       0.6       0.5       0.3       0.9        How old are you?      35+

== 编辑：我把旧答案推到了下面，因为这个问题以前不清楚。

假设您有 2 个调查问题，如下所示：

set.seed(4)
sq_1 <- data.frame(V1 = rnorm(2, .5, .1), V2 = rnorm(2, .5, .1),V3 = rnorm(2, .5, .1),V4 = rnorm(2, .5, .1), row.names=paste0("response",1:2))
sq_2 <- data.frame(V1 = rnorm(2, .5, .1), V2 = rnorm(2, .5, .1),V3 = rnorm(2, .5, .1),V4 = rnorm(2, .5, .1), row.names=paste0("response",1:2))
write.csv(sq_1, "survey_question_1.csv")
write.csv(sq_2, "survey_question_2.csv")

要将它们作为列表读入 R：

files <- list.files(pattern="\\.csv")
survey <- lapply(files, read.csv, header=T, row.names=1)

使用 dplyr 插入问题和响应列：

library(dplyr)    
survey <- lapply(1:length(survey), function(x) survey[[x]] %>% 
               mutate(Question=paste0("Q",x), Response = rownames(.)))
do.call(rbind, survey)


         V1        V2        V3        V4 Question  Response
1 0.5216755 0.5891145 0.6635618 0.3718753       Q1 response1
2 0.4457507 0.5595981 0.5689275 0.4786855       Q1 response2
3 0.6896540 0.5566604 0.5383057 0.5034352       Q2 response1
4 0.6776863 0.5015719 0.4954863 0.5169027       Q2 response2
于 2017-04-28T01:45:59.900 回答