python - foreach html页面中的元素组？

Question

我想从一个类似的网页中抓取一些信息

...
<div class="foo">
    <span class="title">sometext</span>
    <ul class="infos">
        <li class="bar">
            <a class="link" href="...">link1</a>
            <img class="photo" src="..." />
        </li>
        <li class="bar">
            <a class="link" href="...">link2</a>
            <img class="photo" src="..." />
        </li>
        <li class="bar">
            <a class="link" href="...">link3</a>
            <img class="photo" src="..." />
        </li>
    </ul>

    <span class="title">sometext2</span>
    <ul class="infos">
        <li class="bar">
            <a class="link" href="...">link4</a>
            <img class="photo" src="..." />
        </li>
        <li class="bar">
            <a class="link" href="...">link5</a>
            <img class="photo" src="..." />
        </li>
    </ul>

    and so on...

</div>
...

但我不知道如何循环浏览每组信息，以获得一个简单的列表，如

sometext:
- link1   imgsrc
- link2   imgsrc
- link3   imgsrc

sometext2:
- link4   imgsrc
- link5   imgsrc

Answer 1

使用 class 查找所有 span title，找到下一个兄弟并遍历其中的li元素：

from bs4 import BeautifulSoup as Soup

html = """
<div class="foo">
    <span class="title">sometext</span>
    <ul class="infos">
        <li class="bar">
            <a class="link" href="...">link1</a>
            <img class="photo" src="..." />
        </li>
        <li class="bar">
            <a class="link" href="...">link2</a>
            <img class="photo" src="..." />
        </li>
        <li class="bar">
            <a class="link" href="...">link3</a>
            <img class="photo" src="..." />
        </li>
    </ul>

    <span class="title">sometext2</span>
    <ul class="infos">
        <li class="bar">
            <a class="link" href="...">link4</a>
            <img class="photo" src="..." />
        </li>
        <li class="bar">
            <a class="link" href="...">link5</a>
            <img class="photo" src="..." />
        </li>
    </ul>
</div>
"""
soup = Soup(html)

for span in soup.find_all('span', {'class': 'title'}):
    print span.text
    for li in span.next_sibling.next_sibling.find_all('li', {'class': 'bar'}):
        print "- %s % s" % (li.find('a').text, li.find('img').attrs['src'])

印刷：

sometext
- link1 ...
- link2 ...
- link3 ...
sometext2
- link4 ...
- link5 ...