1

我正在尝试通过 browsermob 代理从 https 站点捕获网络数据。

我的 BMP - v2.1.4、firefox - v46.0 和 selenium 服务器 - v2.53.1、rails - v5.1.3。

我的代码如下，

require "rubygems"
require "childprocess"
require 'browsermob/proxy'
require 'selenium-webdriver'
require 'capybara'

@proxy = BrowserMob::Proxy::Client.from("http://#{my_ip}:9090")

client_timeout = Selenium::WebDriver::Remote::Http::Default.new
client_timeout.timeout = 180
client = client_timeout
url = "http://#{my_ip}:4444/wd/hub"

profile     = Selenium::WebDriver::Firefox::Profile.new
proxy_sel   = @proxy.selenium_proxy(:http, :ssl)
profile['browser.cache.disk.enable']    = false
profile['browser.cache.memory.enable']  = false
profile['browser.cache.offline.enable'] = false
profile['network.http.use-cache']       = false

caps = Selenium::WebDriver::Remote::Capabilities.firefox(:firefox_profile => 
profile, :proxy => proxy_sel)
Capybara.register_driver :grid_firefox_proxy do |app|
Capybara::Selenium::Driver.new(
    app,
    :url => url,
    :browser => :remote,
    :http_client => client,
    :desired_capabilities => caps
)
end

Capybara.default_driver = :grid_firefox_proxy
session = Capybara::Session.new(:grid_firefox_proxy)
session.driver.browser.manage.window.maximize
@browser = session

@proxy.new_har
@browser.driver.browser.get(#{https_url})

har = @proxy.har 
puts har.entries.first.request.url
puts har.entries.first.response.status
har.save_to "mean.har"

@proxy.close
@browser.driver.quit

当我运行此代码时，浏览器正在启动并显示“Firefox 无法与服务器建立连接”。在 BMP 服务器中，我收到一个错误，例如，

"[ERROR 2017-10-25T13:18:17,201 org.littleshoot.proxy.impl.
ProxyToServerConnection] (LittleProxy-853-ProxyToServerWorker-2) 
(DISCONNECTED) [id: 0xecabc1af, L:0.0.0.0/0.0.0.0:63045 ! 
R:abcd.com/66.151.240.102:443]: Caught an exception 
on ProxyToServerConnection io.netty.handler.codec.DecoderException: 
javax.net.ssl.SSLHandshakeException: General SSLEngine problem"

当我创建没有 ssl 选项的 selenium 代理时（proxy_sel = @proxy.selenium_proxy(:http)），我能够获取一些不相关的网络条目。

http://ocsp.digicert.com/
http://ocsp.digicert.com/
http://ocsp.digicert.com/
http://ss.symcd.com/
http://clients1.google.com/ocsp
http://ocsp.comodoca.com/
http://ocsp.comodoca.com/
http://ss.symcd.com/
http://clients1.google.com/ocsp
http://ocsp.digicert.com/
http://ocsp.sca1b.amazontrust.com/
http://ocsp.digicert.com/
http://clients1.google.com/ocsp
http://ocsp.digicert.com/
http://ocsp.digicert.com/
http://clients1.google.com/ocsp
http://ocsp.digicert.com/

我想提取正确的网络数据。任何帮助，将不胜感激！！

谢谢！！

4

0 回答 0