我有一系列需要处理的 JSON 消息,并且在字符编码方面遇到了一些问题。我从 RabbitMQ 队列中检索此消息,如下所示:
@data = []
@queue.pop do |metadata, payload|
parsed_message = JSON.parse(payload)
@data << parsed_message
end
然后我使用 Resque 处理它们并将它们排入队列,如下所示:
Resque.enqueue(Worker, @data)
当我运行工人时,它会引发此错误:
** [16:06:13 2012-09-14] 5375: Error reserving job: #<Encoding::InvalidByteSequenceError: "\xC3" on US-ASCII>
** [16:06:13 2012-09-14] 5375: /usr/local/lib/ruby/1.9.1/json/common.rb:148:in `encode'
/usr/local/lib/ruby/1.9.1/json/common.rb:148:in `initialize'
/usr/local/lib/ruby/1.9.1/json/common.rb:148:in `new'
/usr/local/lib/ruby/1.9.1/json/common.rb:148:in `parse'
/usr/local/lib/ruby/gems/1.9.1/gems/multi_json-1.3.6/lib/multi_json/adapters/json_common.rb:7:in `load'
/usr/local/lib/ruby/gems/1.9.1/gems/multi_json-1.3.6/lib/multi_json.rb:93:in `load'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/helpers.rb:38:in `decode'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque.rb:149:in `pop'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/job.rb:100:in `reserve'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque.rb:303:in `reserve'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/worker.rb:209:in `block in reserve'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/worker.rb:207:in `each'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/worker.rb:207:in `reserve'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/worker.rb:136:in `block in work'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/worker.rb:133:in `loop'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/worker.rb:133:in `work'
/usr/local/lib/ruby/gems/1.9.1/gems/resque-1.22.0/lib/resque/tasks.rb:36:in `block (2 levels) in <top (required)>'
/usr/local/lib/ruby/1.9.1/rake/task.rb:205:in `call'
/usr/local/lib/ruby/1.9.1/rake/task.rb:205:in `block in execute'
/usr/local/lib/ruby/1.9.1/rake/task.rb:200:in `each'
/usr/local/lib/ruby/1.9.1/rake/task.rb:200:in `execute'
/usr/local/lib/ruby/1.9.1/rake/task.rb:158:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/usr/local/lib/ruby/1.9.1/rake/task.rb:151:in `invoke_with_call_chain'
/usr/local/lib/ruby/1.9.1/rake/task.rb:144:in `invoke'
/usr/local/lib/ruby/1.9.1/rake/application.rb:116:in `invoke_task'
/usr/local/lib/ruby/1.9.1/rake/application.rb:94:in `block (2 levels) in top_level'
/usr/local/lib/ruby/1.9.1/rake/application.rb:94:in `each'
/usr/local/lib/ruby/1.9.1/rake/application.rb:94:in `block in top_level'
/usr/local/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/1.9.1/rake/application.rb:88:in `top_level'
/usr/local/lib/ruby/1.9.1/rake/application.rb:66:in `block in run'
/usr/local/lib/ruby/1.9.1/rake/application.rb:133:in `standard_exception_handling'
/usr/local/lib/ruby/1.9.1/rake/application.rb:63:in `run'
/usr/local/bin/rake:32:in `<main>'
rake aborted!
我已将工作人员的执行函数中的所有代码包装在开始/救援块中,以尝试捕获异常但无济于事,这使我认为它来自 Resque 内部的某个地方。
我尝试使用原始数据将 Resque 工作人员排队,然后在工作人员内部将其解析为 JSON 并设置 UTF-8 编码:
raw_data.each do |msg|
msg = msg.encode("UTF-8", :invalid => :replace, :undef => :replace, :replace => "")
parsed_msg = JSON.parse(msg) rescue nil
next if parsed_msg.nil?
parsed_data << parsed_msg
end
我仍然得到同样的错误。它并不总是同一个字符,如果我没记错的话,有时它是 \xC2 或 \xD8
有任何想法吗?