ruby - 从 Rails 应用程序调用 Chef 更新命令时出现 Encoding::UndefinedConversionError

Question

我正在构建一个使用 Chef-DK underthehood 的 Rails 应用程序。

我想要实现的是执行与chef update <path/to/policyfile>.

这是代码中有趣的部分：

require 'chef-dk/command/update'

chef_update = ChefDK::Command::Update.new
update_options = [policyfile_path]
update_options << '--debug' if Rails.env.development?
update = chef_update.run(update_options)

运行时，它开始更新策略文件，但失败并显示以下输出：

Attributes already up to date
Building policy k8sworker
Expanded run list: recipe[fail2ban], recipe[os-hardening], recipe[rkhunter], recipe[ssh-hardening], recipe[ssh-keys], recipe[tincvpn]
Caching Cookbooks...
Installing tincvpn ~> 0.1.10 from github
Installing fail2ban 6.0.0
Error: Failed to generate Policyfile.lock
Reason: (Encoding::UndefinedConversionError) "\x8B" from ASCII-8BIT to UTF-8
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `write'
/usr/local/lib/ruby/2.6.0/delegate.rb:349:in `block in delegating_block'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:528:in `block in stream_to_tempfile'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:497:in `call_block'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:488:in `<<'
/usr/local/lib/ruby/2.6.0/net/protocol.rb:163:in `read'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:293:in `block in read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:253:in `inflater'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:283:in `read_body_0'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:204:in `read_body'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:527:in `stream_to_tempfile'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:230:in `block in streaming_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:92:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1518:in `block in transport_request'
/usr/local/lib/ruby/2.6.0/net/http/response.rb:165:in `reading_body'
/usr/local/lib/ruby/2.6.0/net/http.rb:1517:in `transport_request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1479:in `request'
/usr/local/lib/ruby/2.6.0/net/http.rb:1472:in `block in request'
/usr/local/lib/ruby/2.6.0/net/http.rb:920:in `start'
/usr/local/lib/ruby/2.6.0/net/http.rb:1470:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http/basic_client.rb:69:in `request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:370:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:388:in `block (2 levels) in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:485:in `follow_redirect'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:383:in `block in send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:411:in `block in retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `loop'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:409:in `retrying_http_errors'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:365:in `send_http_request'
/usr/local/bundle/gems/chef-15.1.36/lib/chef/http.rb:228:in `streaming_request'
/usr/local/bundle/gems/cookbook-omnifetch-0.8.1/lib/cookbook-omnifetch/artifactserver.rb:42:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile/cookbook_location_specification.rb:81:in `ensure_cached'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:181:in `block in install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `each'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_compiler.rb:176:in `install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:101:in `generate_lock_and_install'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/policyfile_services/install.rb:63:in `run'
/usr/local/bundle/gems/chef-dk-4.0.60/lib/chef-dk/command/update.rb:93:in `run'
/application/app/interactors/node_actions/run_chef_update.rb:24:in `call'
...

这是带有trace日志级别的 Chef 输出：

worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Initiating GET to https://supermarket.chef.io/universe
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:17+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: HTTP 1.1 200 OK
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: cache-control: max-age=0, private, must-revalidate
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: content-encoding: gzip
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: content-type: application/json; charset=utf-8
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:19 GMT
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: etag: W/"461b5134d1467f6950f82af1510078a0"
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: server: nginx
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-content-type-options: nosniff
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-request-id: 0bc2cc4a-0399-4953-93db-699176e11e36
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-runtime: 0.301233
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: transfer-encoding: chunked
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: connection: Close
worker_1     | [2019-07-03T08:02:20+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads.
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response
worker_1     | [2019-07-03T08:02:22+00:00] TRACE: Decompressing gzip response
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Initiating GET to https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:23+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 302 Found
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: cache-control: no-cache
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-type: text/html; charset=utf-8
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:25 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: location: https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: server: nginx
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-content-type-options: nosniff
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-frame-options: SAMEORIGIN
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-request-id: 17a0d07d-cf03-45b3-956f-8eb7fcabd2a7
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-runtime: 0.044161
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-xss-protection: 1; mode=block
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-length: 209
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: connection: Close
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Response Body ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: <html><body>You are being <a href="https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785">redirected</a>.</body></html>
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Response Body -----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: No content-length information collected for the streamed download, cannot identify streamed download.
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::CookieManager#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP calling Chef::HTTP::Decompressor#handle_stream_complete
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Following redirect 1/10
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Initiating GET to https://s3.amazonaws.com/community-files.opscode.com/cookbook_versions/tarballs/28864/original/fail2ban20190508-41006-1kydj0z.tar.gz?1557352785
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Request Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Request Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- HTTP Status and Header Data: ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: HTTP 1.1 200 OK
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-amz-id-2: a2iOnve5iX8paa/pT2s+A21Bm8IvSqbjzYeP8mc2I2kNGSwgj9Wen8IyQlpiQ955grAF7xZKaQE=
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: x-amz-request-id: E74D7A5F59B4B3B0
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: date: Wed, 03 Jul 2019 08:02:26 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: last-modified: Wed, 08 May 2019 21:59:47 GMT
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: etag: "5c15105980cf9bfb2467ed00b789be0a"
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: accept-ranges: bytes
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-type: application/gzip
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content-length: 9020
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: server: AmazonS3
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: connection: close
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: ---- End HTTP Status/Header Data ----
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Streaming download from https://supermarket.chef.io/api/v1/cookbooks/fail2ban/versions/6.0.0/download to tempfile /tmp/chef-rest20190703-1-cbdl7x
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: content_encoding = ''               initializing noop stream deflator.
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::ValidateContentLength::ContentLengthCounter#handle_chunk
worker_1     | [2019-07-03T08:02:25+00:00] TRACE: Chef::HTTP::StreamHandler calling Chef::HTTP::Decompressor::NoopInflater#handle_chunk

因此，就像在对 Amazon S3 中的 fail2ban 存档进行放气时，由于某些编码内容而失败。

从那时起：

然后当我chef update从终端运行命令时，它工作正常，没有错误
当我再次运行我的代码时，它也可以正常工作（我猜是由于厨师缓存了食谱的版本，所以不再经历失败的代码）

所以对我来说，为了让这部分工作，我的代码中好像缺少了一些东西，但我不知道它可能是什么。

我该如何解决这个问题？

PS：该应用程序在带有 Linux Alpine 的 Docker 容器中运行。

score 1 · Accepted Answer

Chef 库正在使用 aTempfile来将下载的数据写入某处，因此我为它的方法设置了别名，以便在将其返回给原始方法之前write调用传递的参数。force_encoding('UTF-8')write

这是我的覆盖（在config/initializers/tempfile_override.rb：

# frozen_string_literal: true

require 'tempfile'

#
# Overrides the Tempfile write method in order to always use `force_encode` with
# UTF-8 in order to solve the issue where `chef update` fails to write
# downloaded cookbooks data and write it to the disk.
# See https://stackoverflow.com/questions/56866746/encodingundefinedconversionerror-when-calling-chef-update-command-from-a-rails
#
class Tempfile
  alias old_write write
  def write(data)
    old_write(data.force_encoding('UTF-8'))
  end
end

我有点被迫这样做，因为我需要更改Chef::UI对象才能在我的应用程序中获取 Chef 日志。

这不干净，但我没有更好的方法来做到这一点，而不必修补厨师本身。

在找到更好的东西之前，这个答案将被接受。

如果您知道更好的方法，请在这里帮助我:)

ruby - 从 Rails 应用程序调用 Chef 更新命令时出现 Encoding::UndefinedConversionError

1 回答 1

Related

Reference