2

我正在尝试从国家气象局下载 GRIB 数据（二进制天气预报数据）。我已经编写了 Python 代码来格式化 HTTP 字符串以获取今天的数据，提前 12 小时。

Python 代码返回 HTTP 字符串，然后尝试使用 urllib.urlopen 下载数据。现在，如果我将 HTTP 字符串粘贴到 Firefox 中，则会下载 GRIB 文件。如果我尝试使用 urllib.urlopen，我会得到以下信息：

Traceback (most recent call last):
File "/home/dantayaga/bovine_aerospace/dev/grib_get.py", line 67, in <module>
webf=urllib.urlopen(griburl)
File "/usr/lib/python2.7/urllib.py", line 86, in urlopen
return opener.open(url)
File "/usr/lib/python2.7/urllib.py", line 207, in open
return getattr(self, name)(url)
File "/usr/lib/python2.7/urllib.py", line 344, in open_http
h.endheaders(data)
File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 776, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 757, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
IOError: [Errno socket error] [Errno -2] Name or service not known

这是我正在使用的 HTTP 字符串：

http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl?file=gfs.t06z.mastergrb2f12&lev_1000_mb=on&lev_975_mb=on&lev_950_mb=on&lev_925_mb=on&lev_900_mb=on&lev_850_mb=on&lev_800_mb=on&lev_750_mb=on&lev_700_mb=on&lev_650_mb=on&lev_600_mb=on&lev_550_mb=上＆lev_500_mb = ON＆lev_450_mb = ON＆lev_400_mb = ON＆lev_350_mb = ON＆lev_300_mb = ON＆lev_250_mb = ON＆lev_200_mb = ON＆lev_150_mb = ON＆lev_100_mb = ON＆lev_70_mb = ON＆lev_30_mb = ON＆lev_20_mb = ON＆lev_10_mb = ON＆var_HGT = ON＆var_RH = ON＆var_TMP = ON＆var_UGRD = ON＆var_VGRD = ON＆var_VVEL = onleftlon = -90rightlon = 90toplat = 90bottomlat-90＆DIR =％ 2Fgfs.2012070706%2Fmaster

如果您在 Firefox 中测试此字符串但它不起作用，请将“20120707”更改为今天的日期，将“06”更改为“00”，它应该可以工作。

我的问题很简单（我认为）：为什么这适用于 Firefox 而不是 urllib？

这是我用来生成 http 字符串然后尝试下载结果的代码：

#Get GRIB files

import urllib

forecast_time='06' #What time the forecast is (00, 06, 12, 18)
forecast_hours='12' #How many hours ahead to forecast (2 or 3 digits)
forecast_date='20120707' #What date the forecast is for yyyymmdd

top_lat=90 #Top of bounding box (North)
bottom_lat=-90 #Bottom of bounding box (South)
left_lon=-90 #Left of bounding box (West)
right_lon=90 #Right of bounding box (East)

griburl='http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl?'
griburl=griburl+'file=gfs.t'+str(forecast_time)+'z.mastergrb2f'
griburl=griburl+forecast_hours

#Select atmospheric levels

griburl=griburl+'&lev_1000_mb=on'  #1000 mb level
griburl=griburl+'&lev_975_mb=on'   #975 mb level
griburl=griburl+'&lev_950_mb=on'   #950 mb level
griburl=griburl+'&lev_925_mb=on'   #925 mb level
griburl=griburl+'&lev_900_mb=on'   #900 mb level
griburl=griburl+'&lev_850_mb=on'   #850 mb level
griburl=griburl+'&lev_800_mb=on'   #800 mb level
griburl=griburl+'&lev_750_mb=on'   #750 mb level
griburl=griburl+'&lev_700_mb=on'   #700 mb level
griburl=griburl+'&lev_650_mb=on'   #650 mb level
griburl=griburl+'&lev_600_mb=on'   #600 mb level
griburl=griburl+'&lev_550_mb=on'   #550 mb level
griburl=griburl+'&lev_500_mb=on'   #500 mb level
griburl=griburl+'&lev_450_mb=on'   #450 mb level
griburl=griburl+'&lev_400_mb=on'   #400 mb level
griburl=griburl+'&lev_350_mb=on'   #350 mb level
griburl=griburl+'&lev_300_mb=on'   #300 mb level
griburl=griburl+'&lev_250_mb=on'   #250 mb level
griburl=griburl+'&lev_200_mb=on'   #200 mb level
griburl=griburl+'&lev_150_mb=on'   #150 mb level
griburl=griburl+'&lev_100_mb=on'   #100 mb level
griburl=griburl+'&lev_70_mb=on'    #70 mb level
griburl=griburl+'&lev_30_mb=on'    #30 mb level
griburl=griburl+'&lev_20_mb=on'    #20 mb level
griburl=griburl+'&lev_10_mb=on'    #10 mb level

#Select variables

griburl=griburl+'&var_HGT=on'  #Height (geopotential m)
griburl=griburl+'&var_RH=on'  #Relative humidity (%)
griburl=griburl+'&var_TMP=on' #Temperature (K)
griburl=griburl+'&var_UGRD=on' #East-West component of wind (m/s)
griburl=griburl+'&var_VGRD=on' #North-South component of wind (m/s)
griburl=griburl+'&var_VVEL=on' #Vertical Windspeed (Pa/s)

#Select bounding box

griburl=griburl+'leftlon='+str(left_lon)
griburl=griburl+'rightlon='+str(right_lon)
griburl=griburl+'toplat='+str(top_lat)
griburl=griburl+'bottomlat'+str(bottom_lat)

#Select date and time

griburl=griburl+'&dir=%2Fgfs.'+forecast_date+forecast_time+'%2Fmaster'
print(griburl)
print('Downloading GRIB file for date '+forecast_date+' time ' +forecast_time + ',    forecasting '+forecast_hours+' hours ahead...')
webf=urllib.urlopen(griburl)
local_filename=forecast_date+'_'+forecast_time+'_'+forecast_hours+'.grib'
localf=open('//home//dantayaga//bovine_aerospace//grib//data//'+local_filename, 'wb')
localf.write(webf.read())
print('Requested grib data written to file '+local_filename)

非常感谢任何帮助。是否存在 Firefox 捕获的格式错误或其他问题？

4

1 回答 1

3

试试这个：

import urllib2
import urllib

url = 'http://nomads.ncep.noaa.gov/cgi-bin/filter_gfs_hd.pl'

forecast_time = '06' #What time the forecast is (00, 06, 12, 18)
forecast_hours = '09' #How many hours ahead to forecast (2 or 3 digits)
forecast_date = '20120705' #What date the forecast is for yyyymmdd

get_parameters = {
    'subregion':'',
    'toplat':90, #Top of bounding box (North)
    'bottomlat':-90, #Bottom of bounding box (South)
    'leftlon':-90, #Left of bounding box (West)
    'rightlon':90, #Right of bounding box (East)
}

get_parameters['file'] = 'gfs.t' + forecast_time + 'z.mastergrb2f' + forecast_hours

on_variables = [
    'lev_1000_mb',
    'lev_975_mb',
    'lev_950_mb',
    'lev_925_mb',
    'lev_900_mb',
    'lev_850_mb',
    'lev_800_mb',
    'lev_750_mb',
    'lev_700_mb',
    'lev_650_mb',
    'lev_600_mb',
    'lev_550_mb',
    'lev_500_mb',
    'lev_450_mb',
    'lev_400_mb',
    'lev_350_mb',
    'lev_300_mb',
    'lev_250_mb',
    'lev_200_mb',
    'lev_150_mb',
    'lev_100_mb',
    'lev_70_mb',
    'lev_30_mb',
    'lev_20_mb',
    'lev_10_mb',

    'var_HGT',  #Height (geopotential m)
    'var_RH',  #Relative humidity (%)
    'var_TMP', #Temperature (K)
    'var_UGRD', #East-West component of wind (m/s)
    'var_VGRD', #North-South component of wind (m/s)
    'var_VVEL' #Vertical Windspeed (Pa/s)
]

get_parameters.update(dict((param, 'on') for param in on_variables))

#Select date and time
get_parameters['dir'] = '/gfs.' + forecast_date + forecast_time + '/master'

print('Downloading GRIB file for date '+forecast_date+' time ' +forecast_time + ',    forecasting '+forecast_hours+' hours ahead...')

req = urllib2.urlopen(url + '?' + urllib.urlencode(get_parameters), timeout = 300) # Theres bug in apache for non-used GET varaibles so we have to manually add them to the url ...
local_filename = forecast_date + '_' + forecast_time + '_' + forecast_hours + '.grib'
local_file = open('/home/dantayaga/bovine_aerospace/grib/data/' + local_filename, 'wb')

local_file.write(req.read())
local_file.close()

print('Requested grib data written to file ' + local_filename)

我已经用另一个文件对其进行了测试，因为您在原始代码中拥有的文件不存在我得到：

数据文件不存在：/pub/data/nccf/com/gfs/prod/gfs.2012070706/master27/gfs.t06z.mastergrb2f12

尝试使用字典、列表、元组或其他复杂的数据结构来存储您的参数，这样您将能够更快地检测到细微的错误，并减少重复代码。

urllib2.urlopen有一个名为datathats 的可选参数用于传递参数，但不幸的是，在 apache 或 python 中都有一个错误，因为我不断得到raise IncompleteRead(value)

httplib：不完整阅读
http://bugs.python.org/issue14044

于 2012-07-08T06:28:43.580 回答