ruby-on-rails - 使用特殊的非 ASCII 字符导入 CSV 文件？

Answer 1

Not sure which version of Ruby you are on. I am assuming it is 1.9.3 or greater.

Try using this to force encoding that your CSV file is saved in

 force_encoding(encoding)

Answer 2

If the values are accessible and reading in from the CSV you can use a helper like this:

  def ascii_only(string)
    encoding_options = {
        :invalid           => :replace,  # Replace invalid byte sequences
        :undef             => :replace,  # Replace anything not defined in ASCII
        :replace           => '',        # Use a blank for those replacements
        :UNIVERSAL_NEWLINE_DECORATOR => true       # Always break lines with \n
    }
    string.encode Encoding.find('ASCII'), encoding_options
  end

Answer 3

I can't duplicate the problem on Ruby 1.9.3 or 2.0. I wrote this test code:

require 'csv'

CSV.foreach('test.csv') do |row|
  puts row
end

And created this test.csv file:

char
™
®

Running the code correctly displays the characters.

Answer 4

What you want to do is write a custom CSV converter, and then reference that converter in your options hash when you call foreach.

Custom converter:

  #define custom converter to eliminate non ASCII characters
  CSV::Converters[:only_ascii] = lambda{|s| 
    begin 
      s.force_encoding("utf-8").encode("utf-8", "binary", :replace => "", :undef => :replace, :invalid => :replace)
    rescue
      s
    end
  }

And then in your options hash:

   CSV.foreach(file.path, headers: true, converters: [:only_ascii]) do |row|
     Yadda yadda
   end

Now, if you want to replace them with reasonable ASCII equivalents instead of blanks, you can write a custom function to do this and then in your custom converter where it says :replace => "", you would do :replace => you_custom_method_name_here.