0

I have the following code:

*** REST OF CODE OMITTED ***
try:
    fullURL = blitzurl + movie
    opener.open(blitzurl)
    urllib2.install_opener(opener)
    request = urllib2.Request(fullURL)
    requestData = urllib2.urlopen(request)
    htmlText = BeautifulSoup(requestData.read())
    
    #panel = htmlText.find(match_class(["panelbox"]))
    #table = htmlText.find("table", {"id" : "scheduletbl"})
    print htmlText

blah....

except Exception, e:
    print str(e)
    print "ERROR: ERROR OCCURED IN MAIN"

I am trying to get the content of a table with id "scheduletbl" (which is inside a div with a class named "panelbox"

the html code looks like this:

*** REST OF CODE OMITTED ***

<div class="panelbox">
<!-- !!!! content here !!!!! -->
<table border="0" cellpadding="2" cellspacing="0" id="scheduletbl" width="95%">
<tr>
<td align="left" colspan="3">
VC = Special Cinema (Velvet Class)<br/>
VS = Special Cinema (Velvet Suite)<br>
DC = Special Cinema (Dining Cinema)<br/>
S = Special Cinema (Satin)<br/>
3D = in RealD 3D<br/>
4DX = 4DX Cinema
</br></td>
</tr>
<tr>
<td class="separator2" colspan="3"><strong>BLITZMEGAPLEX - PARIS VAN JAVA, BANDUNG</strong></td>
</tr>
<tr>
<td colspan="3"><img align="left" height="16" hspace="5" src="../img/ico_rss_schedule_white.gif" width="16"/><strong><a class="navlink" href="../rss/schedule.php">RSS- Paris van Java</a></strong></td>
</tr>
<tr>
<td class="separator"> </td>
<td class="separator" colspan="2">TUESDAY, 24 SEPTEMBER 2013</td>
</tr>
<tr>
<td class="separator"> </td>
<td class="separator" rel="2D" width="20%">
10:30   
</td>
<td class="separator" width="30%">
<a class="navlink" href="https://www.blitzmegaplex.com/olb/seats.php?showdate=2013-09-24&amp;cinema=0100&amp;movie=MOV1954&amp;showtime=10:30&amp;suite=N&amp;movieformat=2D" target="_blank">Buy Tickets</a></td>
</tr></table></div></div>

<tr>
*** and more <tr> tags ***
*** REST OF CODE OMITTED ***

The problem that i am having is that, when i try to extract the content based on the div-id it gets cut off in the middle (i am guessing because improper closing tag).

The thing also happen when i try to extract the content based on (using its id). It will also gets cut off in the middle because there is a , where it is not suppose to be there.

What are the best way to solve this? I have no control about the data since it is scraped from some website.

4

3 回答 3

0

您可以尝试使用https://scraperwiki.com/ - 如果您想检查哪个工具/lib 最适合您完成此任务。

可以选择使用 html5lib、pyquery、bs4 等（简单测试）

你可以试试beautifulsoup：

BeautifulSoup(html).prettify()

html 是你的内容

BS应该擅长处理糟糕的html...

于 2013-09-24T06:05:37.800 回答
0 
re.search(r'id="scheduletbl".+?</table>', page, re.DOTALL)

dotall 如果涉及换行符。这是丑陋的非美丽方式

于 2013-09-24T06:02:10.567 回答
0

如果您使用的是 python 默认包含的解析器，则不正确的结束标记可能会产生问题。正如美丽汤文档中所说：不是很宽松（在 Python 2.7.3 或 3.2.2 之前）。

所以，如果你使用的是之前的版本，你可能会安装 lxml 的 HTML 解析器，它更宽松

$ pip install lxml

或者如果您想要与浏览器相同的 html 解析，您可以安装 html5lib 解析器

$ pip install html5lib

他们可能会更好地解析您的 HTML，并且能够适应错误的标签关闭。Beautiful soup 会自动选择您安装的最佳解析器。

于 2013-09-24T05:11:06.687 回答