0

我正在尝试抓取Forever21，它通过 AJAX 请求加载部分内容。因此，我尝试使用Postman（Chrome 扩展）模拟 AJAX 请求，结果与使用Curl.

这是卷曲请求

curl -X POST -H "Cache-Control: no-cache" -H "Postman-Token: 6d9cdb68-18f7-33c4-003a-3a6940d9d182" -H "Content-Type: application/x-www-form-urlencoded" -d 'ctl00%24MainContent%24ScriptManager1=ctl00%24MainContent%24upColorChart%7Cctl00%24MainContent%24dlColorChart%24ctl01%24imgColorImage&__EVENTARGUMENT=&__ASYNCPOST=true&__VIEWSTATE=' http://www.forever21.com/Product/Product.aspx?BR=f21&Category=outerwear_coats-and-jackets&ProductID=2000102135

这是回应：

69|dataItem||<script type="text/javascript">window.location="about:blank"</script>|55|pageRedirect||/Error/Default.aspx?aspxerrorpath=/Product/Product.aspx|%

这是邮递员变量：

URL: http://www.forever21.com/Product/Product.aspx?BR=f21&Category=outerwear_coats-and-jackets&ProductID=2000102135
Request type: POST (x-www-form-urlencoded)
Variables
     ctl00$MainContent$ScriptManager1: ctl00$MainContent$upColorChart|ctl00$MainContent$dlColorChart$ctl01$imgColorImage
     __EVENTARGUMENT:
     __ASYNCPOST: true
     __VIEWSTATE:

这是回应：

384|updatePanel|ctl00_MainContent_upImageButtonList|

<div id="scroller_frame" style="left:-10000px; position:relative;">
    <ul id="scroller" class="clearfix" style="padding-left: 0px;"></ul>
</div>
                                    |440|updatePanel|ctl00_MainContent_upProductLayout|

<div id="productLayer">
    <a id="ctl00_MainContent_productZoomLink" class="jqzoom" title="" rel="gal1">
        <img id="ctl00_MainContent_productImage" class="ItemImage" title="" src="" style="border-width:0px;padding-left: 0px;" />
    </a>
    <div></div>
</div>
                                    |109|updatePanel|ctl00_MainContent_upColorChart|

                                                                                    |383|updatePanel|ctl00_MainContent_upColorList|

<select name="ctl00$MainContent$ddlColor" onchange="javascript:setTimeout('__doPostBack(\'ctl00$MainContent$ddlColor\',\'\')', 0)" id="ctl00_MainContent_ddlColor" class="input" style="width:175px;"></select>
                                                                                    |298|updatePanel|ctl00_MainContent_upSizeList|

<select name="ctl00$MainContent$ddlSize" id="ctl00_MainContent_ddlSize" class="input" onchange="fnChangeSize();"></select>
                                                                                    |0|hiddenField|__EVENTTARGET||0|hiddenField|__EVENTARGUMENT||0|hiddenField|__LASTFOCUS||1344|hiddenField|__VIEWSTATE|/wEPDwUJNjkyNTA5NjE0Dw9kZBYCZg9kFggCBA8WAh4JaW5uZXJodG1sBQEwZAIFDxYCHwAFBSQwLjAwZAIHDxYCHgRUZXh0BdMFPGRpdiBpZD0iR2xvYmFsTmF2Q29udGFpbmVyIj48ZGl2IGlkPSJHbG9iYWwxIj48YSBocmVmPSJqYXZhc2NyaXB0OnNob3dQb3BXaW4oJ2h0dHA6Ly93d3cuZm9yZXZlcjIxLmNvbS9IdG1scy9odG1sQ29udGVudHMuYXNweD9icj1mMjEmdXJsPS9Qb3B1cC9lbi1VUy8yME9GRlNIT0VTQkFHU19kaXNjbGFpbWVyLmh0bWwnLDQ1MCwxOTAsbnVsbCk7Ij48aW1nIHNyYz0iL2ltYWdlcy9lbi8yMDE0MTEyMi8xLmpwZyIgLz48L2E+PC9kaXY+PGRpdiBpZD0iR2xvYmFsMiI+PGEgaHJlZj0iaHR0cDovL3d3dy5mb3JldmVyMjEuY29tL1Byb2R1Y3QvQ2F0ZWdvcnkuYXNweD9icj1mMjEmY2F0ZWdvcnk9cHJvbW8tc2hvZXMtaGFuZGJhZ3Mtc2FsZSI+PGltZyBzcmM9Ii9pbWFnZXMvZW4vMjAxNDExMjIvMi5qcGciIC8+PC9hPjwvZGl2PjxkaXYgaWQ9Ikdsb2JhbDMiPjxhIGhyZWY9IiMiIG9uQ2xpY2s9InNob3dQb3B1cCgpOyI+PGltZyBzcmM9Ii9pbWFnZXMvZW4vMjAxNDExMjIvMy5qcGciIC8+PC9hPjwvZGl2PjwvZGl2PjxhIGhyZWY9Imh0dHA6Ly93d3cuZm9yZXZlcjIxLmNvbS9Qcm9kdWN0L0NhdGVnb3J5LmFzcHg/YnI9ZjIxJmNhdGVnb3J5PXByb21vLXNob2VzLWhhbmRiYWdzLXNhbGUiIGNsYXNzPSJtdy1nbG9iYWwtYmFyIiBzdHlsZT0iZGlzcGxheTogbm9uZTsiPjxpbWcgZGF0YS1tdy1zcmM9Ii9pbWFnZXMvZW4vMjAxNDExMjIvZ2xvYmFsLmpwZyIgLz48L2E+ZAIKD2QWAmYPZBYCAhYPZBYEZg9kFgJmD2QWAgIBDzwrAAkAZAIBD2QWAmYPZBYCAgEPEGRkFgBkGAEFHl9fQ29udHJvbHNSZXF1aXJlUG9zdEJhY2tLZXlfXxYDBR9jdGwwMCRNYWluQ29udGVudCRpYnRuU2VuZEVtYWlsBRxjdGwwMCRNYWluQ29udGVudCRpYnRuQWRkQmFnBR5jdGwwMCRNYWluQ29udGVudCRpYnRuV2lzaGxpc3Sy0RNtz8zunSVqrWHQd6XGHBYj3g==|8|hiddenField|__VIEWSTATEGENERATOR|3805503D|0|asyncPostBackControlIDs|||0|postBackControlIDs|||164|updatePanelIDs||tctl00$MainContent$upImageButtonList,tctl00$MainContent$upProductLayout,tctl00$MainContent$upColorChart,tctl00$MainContent$upColorList,tctl00$MainContent$upSizeList|0|childUpdatePanelIDs|||159|panelsToRefreshIDs||ctl00$MainContent$upImageButtonList,ctl00$MainContent$upProductLayout,ctl00$MainContent$upColorChart,ctl00$MainContent$upColorList,ctl00$MainContent$upSizeList|2|asyncPostBackTimeout||90|77|formAction||Product.aspx?BR=f21&Category=outerwear_coats-and-jackets&ProductID=2000102135|
4

1 回答 1

2

问题出在“User-Agent”标头上，邮递员没有提到它发送标头。

所以，我附加了 Curl 请求并添加了“User-Agent”标头，它工作正常。

于 2014-11-25T19:44:12.873 回答