python - 有没有一种更 Pythonic 的方式来将两个 HTML 标题行与 colspan 合并？

Question

我在 Python 中使用 BeautifulSoup 来解析一些 HTML。我正在处理的问题之一是我遇到标​​题行之间的 colspan 不同的情况。（标题行是需要组合以获得我的行话中的列标题的行）即一列可能跨越其上方或下方的许多列，并且需要根据跨度附加或前置单词。下面是执行此操作的例程。我使用 BeautifulSoup 拉出 colspan 并拉出每行中每个单元格的内容。longHeader 是包含最多项目的标题行的内容，spanLong 是一个列表，其中包含行中每个项目的跨度。这可行，但看起来不是很 Pythonic。

Alos-如果 diff <0，它就不会工作，我可以用我用来让它工作的相同方法来解决这个问题。但在我这样做之前，我想知道是否有人可以快速查看这个并提出一种更 Pythonic 的方法。我是一名长期的 SAS 程序员，所以我很难打破常规——我会像编写 SAS 宏一样编写代码。

longHeader=['','','bananas','','','','','','','','','','trains','','planes','','','','']
shortHeader=['','','bunches','','cars','','trucks','','freight','','cargo','','all other','','']
spanShort=[1,1,3,1,3,1,3,1,3,1,3,1,3,1,3]
spanLong=[1,1,3,1,1,1,1,1,1,1,1,1,3,1,3,1,3,1,3]
combinedHeader=[]
sumSpanLong=0
sumSpanShort=0
spanDiff=0
longHeaderCount=0

for each in range(len(shortHeader)):
    sumSpanLong=sumSpanLong+spanLong[longHeaderCount]
    sumSpanShort=sumSpanShort+spanShort[each]
    spanDiff=sumSpanShort-sumSpanLong
    if spanDiff==0:
        combinedHeader.append([longHeader[longHeaderCount]+' '+shortHeader[each]])
        longHeaderCount=longHeaderCount+1
        continue
    for i in range(0,spanDiff):
            combinedHeader.append([longHeader[longHeaderCount]+' '+shortHeader[each]])
            longHeaderCount=longHeaderCount+1
            sumSpanLong=sumSpanLong+spanLong[longHeaderCount]
            spanDiff=sumSpanShort-sumSpanLong
            if spanDiff==0:
                combinedHeader.append([longHeader[longHeaderCount]+' '+shortHeader[each]])
                longHeaderCount=longHeaderCount+1
                break

print combinedHeader

Answer 1

这是您的算法的修改版本。zip用于迭代较短的长度和标题，而一个类对象用于计算和迭代较长的项目，以及组合标题。while更适合内循环。（原谅太短的名字）。

class collector(object):
    def __init__(self, header):
        self.longHeader = header
        self.combinedHeader = []
        self.longHeaderCount = 0
    def combine(self, shortValue):
        self.combinedHeader.append(
            [self.longHeader[self.longHeaderCount]+' '+shortValue] )
        self.longHeaderCount += 1
        return self.longHeaderCount

def main():
    longHeader = [ 
       '','','bananas','','','','','','','','','','trains','','planes','','','','']
    shortHeader = [
    '','','bunches','','cars','','trucks','','freight','','cargo','','all other','','']
    spanShort=[1,1,3,1,3,1,3,1,3,1,3,1,3,1,3]
    spanLong=[1,1,3,1,1,1,1,1,1,1,1,1,3,1,3,1,3,1,3]
    sumSpanLong=0
    sumSpanShort=0

    combiner = collector(longHeader)
    for sLen,sHead in zip(spanShort,shortHeader):
        sumSpanLong += spanLong[combiner.longHeaderCount]
        sumSpanShort += sLen
        while sumSpanShort - sumSpanLong > 0:
            combiner.combine(sHead)
            sumSpanLong += spanLong[combiner.longHeaderCount]
        combiner.combine(sHead)

    return combiner.combinedHeader

Answer 2

在这个例子中，你实际上做了很多事情。

1. 您已经“过度处理”了 Beautiful Soup Tag 对象以创建列表。将它们保留为标签。

2. 所有这些类型的合并算法都很难。它有助于处理对称合并的两件事。

这是一个可以直接使用 Beautiful Soup Tag 对象的版本。此外，此版本不假设两行的长度。

def merge3( row1, row2 ):
    i1= 0
    i2= 0
    result= []
    while i1 != len(row1) or i2 != len(row2):
        if i1 == len(row1):
            result.append( ' '.join(row1[i1].contents) )
            i2 += 1
        elif i2 == len(row2):
            result.append( ' '.join(row2[i2].contents) )
            i1 += 1
        else:
            if row1[i1]['colspan'] < row2[i2]['colspan']:
                # Fill extra cols from row1
                c1= row1[i1]['colspan']
                while c1 != row2[i2]['colspan']:
                    result.append( ' '.join(row2[i2].contents) )
                    c1 += 1
            elif row1[i1]['colspan'] > row2[i2]['colspan']:
                # Fill extra cols from row2
                c2= row2[i2]['colspan']
                while row1[i1]['colspan'] != c2:
                    result.append( ' '.join(row1[i1].contents) )
                    c2 += 1
            else:
                assert row1[i1]['colspan'] == row2[i2]['colspan']
                pass
            txt1= ' '.join(row1[i1].contents)
            txt2= ' '.join(row2[i2].contents)
            result.append( txt1 + " " + txt2 )
            i1 += 1
            i2 += 1
    return result

Answer 3

也许看看部分问题的 zip 函数：

>>> execfile('so_ques.py')
[[' '], [' '], ['bananas bunches'], [' '], [' cars'], [' cars'], [' cars'], [' '], [' trucks'], [' trucks'], [' trucks'], [' '], ['trains freight'], [' '], ['planes cargo'], [' '], [' all other'], [' '], [' ']]

>>> zip(long_header, short_header)
[('', ''), ('', ''), ('bananas', 'bunches'), ('', ''), ('', 'cars'), ('', ''), ('', 'trucks'), ('', ''), ('', 'freight'), ('', ''), ('', 'cargo'), ('', ''), ('trains', 'all other'), ('', ''), ('planes', '')]
>>> 

enumerate可以帮助避免使用计数器进行一些复杂的索引：

>>> diff_list = []
>>> for place, header in enumerate(short_header):
    diff_list.append(abs(span_short[place] - span_long[place]))

>>> for place, num in enumerate(diff_list):
    if num:
        new_shortlist.extend(short_header[place] for item in range(num+1))
    else:
        new_shortlist.append(short_header[place])


>>> new_shortlist
['', '', 'bunches', '', 'cars', 'cars', 'cars', '', 'trucks', 'trucks', 'trucks', '',... 
>>> z = zip(new_shortlist, long_header)
>>> z
[('', ''), ('', ''), ('bunches', 'bananas'), ('', ''), ('cars', ''), ('cars', ''), ('cars', '')...

更多的pythonic命名可能会增加清晰度：

    for each in range(len(short_header)):
        sum_span_long += span_long[long_header_count]
        sum_span_short += span_short[each]
        span_diff = sum_span_short - sum_span_long
        if not span_diff:
            combined_header.append...

Answer 4

我想我会回答我自己的问题，但我确实得到了很多帮助。感谢所有的帮助。经过一些小的更正后，我使 S.LOTT 的答案起作用。（它们可能小到不可见（内部笑话））。所以现在的问题是为什么这更 Pythonic？我想我看到它的密度较低/使用原始输入而不是推导/我无法判断它是否更容易阅读--->虽然它很容易阅读

S.LOTT 的回答已更正

row1=headerCells[0]
row2=headerCells[1]

i1= 0
i2= 0
result= []
while i1 != len(row1) or i2 != len(row2):
    if i1 == len(row1):
        result.append( ' '.join(row1[i1]) )
        i2 += 1
    elif i2 == len(row2):
        result.append( ' '.join(row2[i2]) )
        i1 += 1
    else:
        if int(row1[i1].get("colspan","1")) < int(row2[i2].get("colspan","1")):
            c1= int(row1[i1].get("colspan","1"))
            while c1 != int(row2[i2].get("colspan","1")): 
                txt1= ' '.join(row1[i1])  # needed to add when working adjust opposing case
                txt2= ' '.join(row2[i2])     # needed to add  when working adjust opposing case
                result.append( txt1 + " " + txt2 )  # needed to add when working adjust opposing case
                print 'stayed in middle', 'i1=',i1,'i2=',i2, ' c1=',c1
                c1 += 1
                i1 += 1    # Is this the problem it
           
        elif int(row1[i1].get("colspan","1"))> int(row2[i2].get("colspan","1")):
                # Fill extra cols from row2  Make same adjustment as above
            c2= int(row2[i2].get("colspan","1"))
            while int(row1[i1].get("colspan","1")) != c2:
                result.append( ' '.join(row1[i1]) )
                c2 += 1
                i2 += 1
        else:
            assert int(row1[i1].get("colspan","1")) == int(row2[i2].get("colspan","1"))
            pass
       
    
        txt1= ' '.join(row1[i1])
        txt2= ' '.join(row2[i2])
        result.append( txt1 + " " + txt2 )
        print 'went to bottom', 'i1=',i1,'i2=',i2
        i1 += 1
        i2 += 1
print result

Answer 5

好吧，我现在有一个答案。我正在考虑这个问题，并决定我需要使用每个答案的一部分。我仍然需要弄清楚我想要一个类还是一个函数。但我有我认为可能比其他任何算法都更 Pythonic 的算法。但是，它大量借鉴了一些非常慷慨的人提供的答案。我非常感谢这些，因为我学到了很多东西。

为了节省编写测试用例的时间，我将在 IDLE 中粘贴我一直在使用的完整代码，并在其后添加一个 HTML 示例文件。除了对类/函数做出决定（并且我需要考虑如何在我的程序中使用此代码）之外，我很高兴看到任何使代码更加 Pythonic 的改进。

from BeautifulSoup import BeautifulSoup

original=file(r"C:\testheaders.htm").read()

soupOriginal=BeautifulSoup(original)
all_Rows=soupOriginal.findAll('tr')


header_Rows=[]
for each in range(len(all_Rows)):
    header_Rows.append(all_Rows[each])


header_Cells=[]
for each in header_Rows:
    header_Cells.append(each.findAll('td'))

temp_Header_Row=[]
header=[]
for row in range(len(header_Cells)):
    for column in range(len(header_Cells[row])):
        x=int(header_Cells[row][column].get("colspan","1"))
        if x==1:
            temp_Header_Row.append( ' '.join(header_Cells[row][column]) )

        else:
            for item in range(x):

                temp_Header_Row.append( ''.join(header_Cells[row][column]) )

    header.append(temp_Header_Row)
temp_Header_Row=[]
combined_Header=zip(*header)

for each in combined_Header:
    print each

好的测试文件内容如下对不起我试图附上这些但无法实现：

  <TABLE style="font-size: 10pt" cellspacing="0" border="0" cellpadding="0" width="100%">
  <TR valign="bottom">
  <TD width="40%">&nbsp;</TD>
  <TD width="5%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>

  <TD width="5%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>

  <TD width="5%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>

  <TD width="5%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>

  <TD width="5%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="3%">&nbsp;</TD>
  <TD width="1%">&nbsp;</TD>
  </TR>
  <TR style="font-size: 10pt" valign="bottom">
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">FOODS WE LIKE</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">&nbsp;</TD>
  <TD>&nbsp;</TD>
  </TR>
  <TR style="font-size: 10pt" valign="bottom">
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="CENTER" colspan="6">SILLY STUFF</TD>

  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">OTHER THAN</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="CENTER" colspan="6">FAVORITE PEOPLE</TD>
  <TD>&nbsp;</TD>
  </TR>
  <TR style="font-size: 10pt" valign="bottom">
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">MONTY PYTHON</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">CHERRYPY</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">APPLE PIE</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">MOTHERS</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">FATHERS</TD>
  <TD>&nbsp;</TD>
  </TR>
  <TR style="font-size: 10pt" valign="bottom">
  <TD nowrap align="left">Name</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">SHOWS</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">PROGRAMS</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">BANANAS</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">PERFUME</TD>
  <TD>&nbsp;</TD>
  <TD>&nbsp;</TD>
  <TD nowrap align="right" colspan="2">TOOLS</TD>
  <TD>&nbsp;</TD>
  </TR>
  </TABLE>