我们有一个代码,它从我们的 tomcat 服务器中提取服务列表并将其插入到数据库中。基本上,代码将从具有以下结构的 html 结果中接收大量输出(100+ 个结果):
<TABLE bgcolor=#dddddd border=1>
<TR>
<TD valign="top"><B>name</B></TD>
<TD>Loader</TD>
</TR>
<TR>
<TD valign="top"><B>enabled</B></TD>
<TD>true</TD>
</TR>
<TR>
<TD valign="top"><B>loadok</B></TD>
<TD>13</TD>
</TR>
</TABLE>
<TABLE bgcolor=#dddddd border=1>
<TR>
<TD valign="top"><B>name</B></TD>
<TD>tester</TD>
</TR>
<TR>
<TD valign="top"><B>enabled</B></TD>
<TD>false</TD>
</TR>
<TR>
<TD valign="top"><B>loadok</B></TD>
<TD>13</TD>
</TR>
</TABLE>
以下是实际代码(有效)
#!/usr/bin/env python
import psycopg2
import urllib2
import base64
import sys
import re
import lxml.html as LH
con = None
try:
con = psycopg2.connect(database='xx', user='xx',password='xx',host='vxx')
cur = con.cursor()
qry ="select iservers.env,iservers.family,iservers.prefix,iservers.iserver,iservers.login,iservers.password,services.service," +\
"proxy_access.proxy_server,proxy_access.proxy_user,proxy_access.proxy_pass " +\
"from services,iservers,proxy_access where iservers.env='TEST' and services.id='2' "
cur.execute(qry)
data = cur.fetchall()
for result in data:
env = result[0]
family = result[1]
prefix = result[2]
iserver = result[3]
login = result[4]
password = result[5]
service = result[6]
proxyHost = result[7]
proxyUser = result[8]
proxyPass = result[9]
proxy_auth = "http://"+proxyUser+":"+proxyPass+"@"+proxyHost
proxy_handler = urllib2.ProxyHandler({"http": proxy_auth})
opener = urllib2.build_opener(proxy_handler)
urllib2.install_opener(opener)
request = urllib2.Request("http://"+iserver+service)
base64string = base64.encodestring('%s:%s' % (login, password)).replace('\n', '')
request.add_header("Authorization", "Basic %s" % base64string)
response = urllib2.urlopen(request)
html = response.read()
###################### CHANGE THIS TO USE A HTML PARSER
regex = r"name</B></TD>\s<TD>(.*?)</TD>\s</TR>\s<TR>\s(.*)enabled</B></TD>\s<TD>(.*)</TD>"
for m in re.finditer(regex,html):
print "inserting\t"+iserver+"\t"+m.group(1)
cur.execute("INSERT INTO pereirtc.package_status (env,family,iserver,prefix,package,status) values (%s,%s,%s,%s,%s,%s)",(env,family,iserver,prefix,m.group(1),m.group(3)))
con.commit()
###################### END
except psycopg2.DatabaseError, e:
print 'Error %s' % e
sys.exit(1)
finally:
if con:
con.close()
在 Stack Overflow 的帮助下,有人建议我将“更改此 ....”的块更改为 libxml。所以我得到了以下块来使用:
doc = LH.fromstring(html)
tds = (td.text_content() for td in doc.xpath("//td"))
for td, val in zip(*[tds]*2):
if td in ("name","enabled"):
print (td,val)
通过上面的示例,我得到了结果:
('name', 'Loader')
('enabled', 'true')
我想要的是,使用 xpath 将结果插入到数据库中。由于我从 python 开始,我无法使用 xpath/libxml 来做到这一点。