1

问题：

我有一个从以压缩 CSV 文件格式存储的 S3 存储桶中提取的数据集。数据来自 S3 API 作为原始向量，然后我将其保存为文件，然后从该文件中读取数据。

有没有一种方法可以直接从原始向量中读取数据而无需编写这些临时文件？

当前代码：

# Import packages -------------------------------------------------------------
library(paws)

# Set up S3 access ------------------------------------------------------------
s3 <- paws::s3()
aws_s3_bucket <- Sys.getenv("AWS_S3_BUCKET")

# Fetch dataset from S3 which returns a raw vector ----------------------------
s3_object <- s3$get_object(Bucket = aws_s3_bucket, Key = "data.csv.gz")
s3_object_body <- s3_object$Body

# Write the raw vector to a temporary file (This is what I want to remove) ----
file_name <- "s3_files/mydataset.csv.gz"
if (file.exists(file_name)) { unlink(file_name) }
writeBin(s3_object_body, con = file_name)

# Finally, read the data from the file ----------------------------------------
data <- data.table::fread(file_name)

一些没有奏效的尝试：

尝试readr::read_csv(s3_object_body)导致以下错误：

Error in vroom_(file, delim = delim %||% col_types$delim, col_names = col_names,  : 
  embedded nul in string: '\037<U+008B>\b\b<f1>7\aa\002<ff>data.csv'

尝试iotools::read.csv.raw(s3_object_body)导致以下错误：

Error in isOpen(con, "rb") : unimplemented type 'raw' in 'asInteger'
4

0 回答 0