0

我无法将 XML 获取到 python 数据框

您能帮我将 XML 解析为 python 数据框吗？我似乎无法让它工作这是我走了多远：

import xmltodict 
import pandas as pd
import requests
from bs4 import BeautifulSoup
 def get_xml():
    url="http://energywatch.natgrid.co.uk/EDP-PublicUI/PublicPI/InstantaneousFlowWebService.asmx"
    headers = {'content-type': 'application/soap+xml; charset=utf-8'}
    body ="""<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
                <soap12:Body>
                <GetInstantaneousFlowData xmlns="http://www.NationalGrid.com/EDP/UI/" />
                </soap12:Body>
                </soap12:Envelope>"""

    response = requests.post(url,data=body,headers=headers)
    return response.content

response = get_xml()
soup = BeautifulSoup(response, 'lxml')
table_columns = []
for item in soup.find_all(['EDPObjectName'.lower()]):
    table_columns.append(item.text)
table_columns=pd.DataFrame(table_columns)
table_rows=[]
for item in soup.find_all(['applicableat']):
    table_rows.append(item.text) 
df1=pd.DataFrame(table_rows).drop_duplicates() 
#df1=pd.to_datetime(df1)
table=[]
for item in soup.find_all(['flowrate']):
    table.append(item.text) 
df=pd.DataFrame(table)
 df_final=pd.DataFrame(df, columns=table_columns, index=df1)

这是我正在寻找的结果：

                    ALDBROUGH   AVONMOUTH   BACTON BBL  …
    2019-08-08T13:00:00 0       1.23    5.1         …
    2019-08-08T13:02:00 0       1.23    5.1         …
    2019-08-08T13:04:00 0       3.23    5.1         …
    2019-08-08T13:06:00 0       3.23    5.1         …
    2019-08-08T13:08:00 0       3.23    5.23            …
    2019-08-08T13:10:00 0       4.23    5.204           …
4

2 回答 2

0

这个问题与其他 xml 解析问题非常相似，因为你有一个分层的数据结构，你需要将它展平。我提出的解决方案将时间戳、位置和流量转换为列，并使每个日志条目成为一行。我还遵循简约原则，即我尝试以某种方式解析 xml，以便我的展平数据已经采用最容易转换为数据框的格式。变量“数据”具有字典的结构，每一列都有一个键。字典中的值是数据列表，列表中的每个位置表示条目属于哪一行：

import pandas as pd
import requests
from bs4 import BeautifulSoup
def get_xml():
    url="http://energywatch.natgrid.co.uk/EDP-PublicUI/PublicPI/InstantaneousFlowWebService.asmx"
    headers = {'content-type': 'application/soap+xml; charset=utf-8'}
    body ="""<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope"><soap12:Body><GetInstantaneousFlowData xmlns="http://www.NationalGrid.com/EDP/UI/" /></soap12:Body></soap12:Envelope>"""
    response = requests.post(url,data=body,headers=headers)
    return response.content

response = get_xml()
soup = BeautifulSoup(response, 'lxml')

data = {'timestamp':[], 'place':[], 'flowrate':[]}

for group in soup.find_all('edpobjectbe'):
    place = group.find('edpobjectname').text
    for xml in group.find_all('edpenergydatabe'):
        data['place'].append(place)
        data['timestamp'].append(xml.find('applicableat').text)
        data['flowrate'].append(xml.find('flowrate').text)

df = pd.DataFrame(data)
df

请注意，我正在对父元素 'edpenergydatabe' 执行 find_all()，因此我可以确定时间戳适用于与其关联的所有条目。如果您想要不同的行和列排列，您现在可以使用像 transpose() 这样的 pandas 函数来实现。我希望这可以帮助您走上正轨！

于 2019-08-08T14:20:50.563 回答
0

尝试使用：

from bs4 import BeautifulSoup
import pandas as pd

name_list = []

prev_df = pd.DataFrame(columns=['time'])
response = BeautifulSoup(get_xml(), 'lxml')
for x in response.find_all('edpobjectbe'):
    list_small = list()

    name = str(x.find('edpobjectname').text).strip()
    name_list.append(name)

    data = x.find_all('edpenergydatabe')
    print(name)
    for y in data:

        applicableat = str(y.find('applicableat').text).strip()
        flowrate = str(y.find('flowrate').text).strip()

        list_small.append([applicableat, flowrate])
    df = pd.DataFrame(list_small, columns=['time', name])
    prev_df = pd.DataFrame.merge(prev_df, df, how='right', on='time')
print(prev_df)

检查这是否适合您！

于 2019-08-08T14:43:04.260 回答