ruby-on-rails - Ruby on rails：create_fixtures 内存问题

Question

我必须创建大约 10 万条记录。这些记录在 csv 文件中，并且正在使用 create_fixtures 函数加载。在我的开发机器上它很慢，但它完成了。问题从生产开始，每个进程都有内存限制，这会导致终止 rake 进程。我认为这是因为 create_fixtures 正在将所有数据导入内存。有谁知道如何强制它导入较小的块（在我将一个大的 csv 切成几个较小的块之前）。

Answer 1

Don't do it!

create_fixtures is designed for loading test data, which should be only as big as needed to exercise a feature. It's not intended for loading thousands of records onto a production (or any other kind of) database. If it's a one-off then maybe OK, but as a regular thing it would make me very nervous.

If your data is simple enough, by which I mean a simple String#split would work, then that should probably be your approach, something like

File.foreach(csv_file_path) do |line|
  fields = line.split(/,/)
  # create records from the array of fields
end

Otherwise (i.e. you may have string values with quotes or commas, missing field values or multiple record formats, that sort of thing) you should probably look at the CSV library, which is already part of the Ruby 1.8.6 install, or better yet, look at the FasterCSV Gem, which replaces CSV in 1.9 onwards.

UPDATE: Handily, Ryan Bates just posted a screencast on the vexatious topic of seed data...

Answer 2

您如何加载/解析 CSV？我想我会使用 Ruby 的 File utils 自己打开、读取和解析每一行。