0

我正在尝试自动化一些东西，但最近一个网站设置了一个协议障碍，需要达成一致才能访问其他页面和文件。

https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/DataExports/DataExports.aspx

我无法使用下面的 java 后查询代码绕过它，我得到的只是该协议障碍页面的 html 内容：

HttpClient httpClient = new HttpClient();
        httpClient.getParams().setParameter(
                ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true);

        PostMethod postMethod = new PostMethod(
                "https://www.paoilandgasreporting.state.pa.us/publicreports/Modules/Welcome/Welcome.aspx") {
            @Override
            public boolean getFollowRedirects() {
                return true;
            }
        };

        postMethod.addParameter("ctl00$MainContent$AgreeButton", "Agree");



        try {
            httpClient.executeMethod(postMethod);
        } catch (HttpException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

        if (postMethod.getStatusCode() == HttpStatus.SC_OK) {
            String resp = postMethod.getResponseBodyAsString();
            System.out.println(resp);
        } else {
            // ...postMethod.getStatusLine();
        }

我使用 firebug 获得了 post 参数，即使我尝试添加其他两个 post 参数，即使这也无济于事

//postMethod.addParameter("__EVENTVALIDATION","%2FwEWAwLA5%2FC9CwLuhr%2FJBgKws9WDC7O5VJxkt6T7bUFREi%2BvwFPh%2FDo37JKON9tdbF4jnCXF");
//  postMethod.addParameter ("__VIEWSTATE","%2FwEPDwUKMTAxODQ0NzY0Nw9kFgJmD2QWAgIDD2QWBAIBDxYCHgRUZXh0BdgQPGRpdiBzdHlsZT0iaGVpZ2h0OiAxNDlweDsgd2lkdGg6IDk1MnB4OyBiYWNrZ3JvdW5kLXJlcGVhdDogbm8tcmVwZWF0OyBiYWNrZ3JvdW5kLWltYWdlOnVybChodHRwczovL3d3dy5haHMyLmRlcC5zdGF0ZS5wYS51cy9pbWFnZXMvZGVwX2V4dGVybmFsX21haW5fYmFubmVyXzIuanBnKSI%2BCgoJPGEgc3R5bGU9ImhlaWdodDoxMDBweDsgd2lkdGg6MzkwcHg7IGxlZnQ6MHB4OyB0b3A6MHB4OyBwb3NpdGlvbjpyZWxhdGl2ZTsgZGlzcGxheTpibG9jazsgdGV4dC1kZWNvcmF0aW9uOm5vbmU7IiB0aXRsZT0iREVQIEhvbWUiIGhyZWY9Imh0dHA6Ly93d3cuZGVwd2ViLnN0YXRlLnBhLnVzLyI%2BCgkJPGVtIHN0eWxlPSJ2aXNpYmlsaXR5OiBoaWRkZW47IGNvbG9yOiAjMDAwMDAwOyBiYWNrZ3JvdW5kLWNvbG9yOiAjRkZGRkZGOyI%2BQWdlbmN5PC9lbT4KCTwvYT4KCTxhIHN0eWxlPSJoZWlnaHQ6MzBweDsgd2lkdGg6MTYwcHg7IGxlZnQ6OHB4OyB0b3A6MTJweDsgcG9zaXRpb246cmVsYXRpdmU7IGRpc3BsYXk6YmxvY2s7IHRleHQtZGVjb3JhdGlvbjpub25lOyIgdGl0bGU9IlBlbm5zeWx2YW5pYSIgaHJlZj0iaHR0cDovL3BhLmdvdi8iPgoJCTxlbSBzdHlsZT0idmlzaWJpbGl0eTogaGlkZGVuOyBjb2xvcjogIzAwMDAwMDsgYmFja2dyb3VuZC1jb2xvcjogI0ZGRkZGRjsiPlBlbm5zeWx2YW5pYTwvZW0%2BCgk8L2E%2BCgoJPGEgc3R5bGU9ImhlaWdodDoxNnB4OyB3aWR0aDoxNDBweDsgbGVmdDo2MjBweDsgdG9wOi0xOHB4OyBwb3NpdGlvbjpyZWxhdGl2ZTsgZGlzcGxheTpibG9jazsgdGV4dC1kZWNvcmF0aW9uOm5vbmU7IHRleHQtYWxpZ246Y2VudGVyOyBjb2xvcjogI0ZGRkZGRjsgYmFja2dyb3VuZC1jb2xvcjogIzE2M0Y2RDsgZm9udC1zaXplOiAwLjhlbTsgZm9udC13ZWlnaHQ6IGJvbGQ7IGZvbnQtZmFtaWx5OiBWZXJkYW5hLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyIgaHJlZj0iaHR0cDovL3d3dy5wb3J0YWwuc3RhdGUucGEudXMvcG9ydGFsL3NlcnZlci5wdD9vcGVuPTUxMiZvYmpJRD0yMDU5OCZQYWdlSUQ9MTA1MjE2MCZtb2RlPTIiPgoJCVBBIFNUQVRFIEFHRU5DSUVTCgk8L2E%2BCgoJPGEgc3R5bGU9ImhlaWdodDoxNnB4OyB3aWR0aDoxNjBweDsgbGVmdDo3ODBweDsgdG9wOi0zNHB4OyBwb3NpdGlvbjpyZWxhdGl2ZTsgZGlzcGxheTpibG9jazsgdGV4dC1kZWNvcmF0aW9uOm5vbmU7IHRleHQtYWxpZ246Y2VudGVyOyBjb2xvcjogI0ZGRkZGRjsgYmFja2dyb3VuZC1jb2xvcjogIzE2M0Y2RDsgZm9udC1zaXplOiAwLjhlbTsgZm9udC13ZWlnaHQ6IGJvbGQ7IGZvbnQtZmFtaWx5OiBWZXJkYW5hLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyIgaHJlZj0iaHR0cDovL3d3dy5wb3J0YWwuc3RhdGUucGEudXMvcG9ydGFsL3NlcnZlci5wdD9vcGVuPTUxMiZvYmpJRD0yMDU5OCZQYWdlSUQ9MTA1MjE1OCZtb2RlPTIiPgoJCU9OTElORSBTRVJWSUNFUwoJPC9hPgoKCTxhIHN0eWxlPSJoZWlnaHQ6MTZweDsgd2lkdGg6MTYwcHg7IGxlZnQ6NjE1cHg7IHRvcDotMjlweDsgcG9zaXRpb246cmVsYXRpdmU7IGRpc3BsYXk6YmxvY2s7IHRleHQtZGVjb3JhdGlvbjpub25lOwl0ZXh0LWFsaWduOmNlbnRlcjsgY29sb3I6ICNGRkZGRkY7IGZvbnQtc2l6ZTogMC44ZW07IGZvbnQtZmFtaWx5OiBWZXJkYW5hLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyIgaHJlZj0iaHR0cDovL3d3dy5nb3Zlcm5vci5wYS5nb3YiIHRpdGxlPSJUb20gQ29yYmV0dCwgR292ZXJub3IiPlRvbSBDb3JiZXR0LCBHb3Zlcm5vcgoJPC9hPgoJPGEgc3R5bGU9ImhlaWdodDoxNnB4OyB3aWR0aDoxOTBweDsgbGVmdDo3NjVweDsgdG9wOi00NXB4OyBwb3NpdGlvbjpyZWxhdGl2ZTsgZGlzcGxheTpibG9jazsgdGV4dC1kZWNvcmF0aW9uOm5vbmU7CXRleHQtYWxpZ246Y2VudGVyOyBjb2xvcjogI0ZGRkZGRjsgZm9udC1zaXplOiAwLjhlbTsgZm9udC1mYW1pbHk6IFZlcmRhbmEsIEFyaWFsLCBIZWx2ZXRpY2EsIHNhbnMtc2VyaWY7IiBocmVmPSJodHRwOi8vd3d3LmRlcHdlYi5zdGF0ZS5wYS51cy9zZWNyZXRhcnkiIHRpdGxlPSJNaWNoYWVsIEtyYW5jZXIsIFNlY3JldGFyeSI%2BTWljaGFlbCBLcmFuY2VyLCBTZWNyZXRhcnkKCTwvYT4KCQkKPC9kaXY%2BZAIFDxYCHwAFxgg8ZGl2IHN0eWxlPSJ3aWR0aDogMTAwJTsgdGV4dC1hbGlnbjogY2VudGVyOyBtYXJnaW46IDAgYXV0bztmb250LXNpemU6IDAuOWVtOyBsaW5lLWhlaWdodDogMTZweDsgY29sb3I6ICMwMzM2Njg7IGZvbnQtZmFtaWx5OiBWZXJkYW5hLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGZvbnQtdmFyaWFudDogbm9ybWFsOyB0ZXh0LWRlY29yYXRpb246IG5vbmU7Ij4KCTxhIHN0eWxlPSJmb250LXNpemU6IDAuOWVtOyBsaW5lLWhlaWdodDogMTZweDsgY29sb3I6ICMwMzM2Njg7IGZvbnQtZmFtaWx5OiBWZXJkYW5hLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGZvbnQtdmFyaWFudDogbm9ybWFsOyB0ZXh0LWRlY29yYXRpb246IG5vbmU7Imh0dHA6Ly93d3cucG9ydGFsLnN0YXRlLnBhLnVzL3BvcnRhbC9zZXJ2ZXIucHQ%2Fb3Blbj01MTImb2JqSUQ9MjA1OTgmUGFnZUlEPTEwNTE2MDYmbW9kZT0yIgoJdGl0bGU9IlByaXZhY3kgUG9saWN5Ij5Qcml2YWN5IFBvbGljeTwvYT4KCSZuYnNwO3wmbmJzcDsKCTxhIHN0eWxlPSJmb250LXNpemU6IDAuOWVtOyBsaW5lLWhlaWdodDogMTZweDsgY29sb3I6ICMwMzM2Njg7IGZvbnQtZmFtaWx5OiBWZXJkYW5hLCBBcmlhbCwgSGVsdmV0aWNhLCBzYW5zLXNlcmlmOyBmb250LXN0eWxlOiBub3JtYWw7IGZvbnQtd2VpZ2h0OiBub3JtYWw7IGZvbnQtdmFyaWFudDogbm9ybWFsOyB0ZXh0LWRlY29yYXRpb246IG5vbmU7ImhyZWY9Imh0dHA6Ly93d3cucG9ydGFsLnN0YXRlLnBhLnVzL3BvcnRhbC9zZXJ2ZXIucHQ%2Fb3Blbj01MTImb2JqSUQ9MjA1OTgmUGFnZUlEPTEwNTE5NDAmbW9kZT0yIgoJdGl0bGU9IlNlY3VyaXR5IFBvbGljeSI%2BU2VjdXJpdHkgUG9saWN5PC9hPgkJCgk8YnIgLz4KCTxzcGFuIHN0eWxlPSJmb250LXNpemU6IDAuOWVtIj4KCQlDb3B5cmlnaHQmbmJzcDsmY29weTsyMDA5LTIwMTImbmJzcDtDb21tb253ZWFsdGggb2YgUGVubnN5bHZhbmlhLiBBbGwgUmlnaHRzIFJlc2VydmVkCgk8L3NwYW4%2BCjwvZGl2PmRkxyei%2F5tm7ky3D270eOJqk5kA63kYz4W0lQuUC%2BjoMj4%3D");

关于如何绕过这些页面的任何想法？

谢谢你的帮助。

4

1 回答 1

0

如果您查看协议页面的 HTML，您会看到“同意”按钮所在的表单标记提交到相对 URL“Agreement.aspx”。本质上，这会在服务器上设置某种 cookie 或会话变量，告诉服务器不要再将那个页面贴在你的脸上。

您必须使用相同的 HTTPClient 实例来执行这两个步骤：

  • 同意，提交到 Agreement.aspx
  • 然后继续你之前已经在做的 POST -

为了在 HTTPClient 中获得会话跟踪，您需要进行一些不同的设置：

HttpClient httpClient = new DefaultHttpClient();
CookieStore cookieStore = new BasicCookieStore();
HttpContext httpContext = new BasicHttpContext();
httpContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);

然后将此 httpContext 实例传递给每个请求：

HttpResponse resp = httpClient.executeMethod(postMethod, httpContext);
于 2012-11-09T20:15:06.420 回答