1

我正在尝试通过此代码获取特定公司的关键财务数据（以下代码中的股票）：

        netIncomeAr = []

        endLink = 'order=asc'   # order=asc&
        try:

            netIncome = urllib.request.urlopen('https://www.quandl.com/api/v3/datasets/RAYMOND/'+stock.upper()+'_NET_INCOME_A.csv?'+endLink).read()

            splitNI = netIncome.split('\n')
            print('Net Income:')
            for eachNI in splitNI[1:-1]:
                print(eachNI)
                netIncomeAr.append(eachNI)


            incomeDate, income = np.loadtxt(netIncomeAr, delimiter=',',unpack=True,
                                            converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

        except Exception as e:
            print('failed in the Quandl grab')
            print(str(e))
            time.sleep(555)

但是我收到了我设计的“Quandl 抓取失败”的错误消息。我知道错误必须在 Quandl 执行 urllib.request 的第一行。

有谁知道为什么这段代码不起作用？

好的 - 谢谢罗兰，

我已将我的代码更改为这个有限的概念验证片段：

import urllib.request, urllib.error, urllib.parse
import time
import datetime
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.ticker as mticker
import matplotlib.dates as mdates

evenBetter = ['GOOG','AAPL']


def graphData(stock, MA1, MA2):
    #######################################
    #######################################
    '''
        Use this to dynamically pull a stock from Quandl:
    '''
    print('Currently Pulling',stock)

    netIncomeAr = []
#    revAr = []
#    ROCAr = []

    endLink = 'order=asc'

    netIncome = str(urllib.request.urlopen('https://www.quandl.com/api/v3/datasets/RAYMOND/'+stock.upper()+'_NET_INCOME_A.csv?'+endLink).read())[2:-1]
    # convert to string, remove leading "b'" and trailing "'" characters.
    # netIncome = 'head\\ndata\\ndata\\n...'


    splitNI = netIncome.split('\\')[1:-1]
    # data segments still have leading 'n' character.
    # the [1:-1] is more pythonic and releases memory.
    for i in range (len(splitNI)):
        splitNI[i] = splitNI[i][1:]
    # data segments are now converted.

    print('Net Income:')
    for eachNI in splitNI:
        print(eachNI)
        netIncomeAr.append(eachNI)


    incomeDate, income = np.loadtxt(netIncomeAr, delimiter=',',unpack=True,
                                    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

for stock in evenBetter:
    graphData(stock,25,50)

现在我将 urllib.request 问题转移到另一个问题......以下错误：

Currently Pulling GOOG
Net Income:
2009-12-31,6520448000.0
2010-12-31,8505000000.0
2011-12-31,9737000000.0
2012-12-31,10737000000.0
2013-12-31,12920000000.0
Traceback (most recent call last):

  File "<ipython-input-3-5ce0b8405254>", line 1, in <module>
    runfile('C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py', wdir='C:/Users/Morten/Google Drev/SpyderProject/test')

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 57, in <module>
    graphData(stock,25,50)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 54, in graphData
    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 860, in loadtxt
    items = [conv(val) for (conv, val) in zip(converters, vals)]

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 860, in <listcomp>
    items = [conv(val) for (conv, val) in zip(converters, vals)]

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\matplotlib\dates.py", line 261, in __call__
    return date2num(datetime.datetime(*time.strptime(s, self.fmt)[:6]))

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\_strptime.py", line 494, in _strptime_time
    tt = _strptime(data_string, format)[0]

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\_strptime.py", line 306, in _strptime
    raise TypeError(msg.format(index, type(arg)))

TypeError: strptime() argument 0 must be str, not <class 'bytes'>

根据 Davse Bamse 的建议，我看到了以下回溯（这是一个艰难的回溯）：

Currently Pulling GOOG
Net Income:
Traceback (most recent call last):

  File "<ipython-input-3-c3f1db0f3995>", line 1, in <module>
    runfile('C:/Users/Morten/Google Drev/SpyderProject/test/sentdex_Test_comp_screener_own_webscraper2.py', wdir='C:/Users/Morten/Google Drev/SpyderProject/test')

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/sentdex_Test_comp_screener_own_webscraper2.py", line 59, in <module>
    graphData(stock)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/sentdex_Test_comp_screener_own_webscraper2.py", line 56, in graphData
    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 845, in loadtxt
    converters[i] = conv

IndexError: list assignment index out of range

使用 Davse Bamse 的新建议，在转换器中包含这样的列表：

[incomeDate, income] = np.loadtxt(netIncomeAr, delimiter=',',unpack=True,
                                converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

我看到这个错误：

Currently Pulling GOOG
Net Income:
C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py:823: UserWarning: loadtxt: Empty input file: "[]"
  warnings.warn('loadtxt: Empty input file: "%s"' % fname)
Traceback (most recent call last):

  File "<ipython-input-1-c3f1db0f3995>", line 1, in <module>
    runfile('C:/Users/Morten/Google Drev/SpyderProject/test/sentdex_Test_comp_screener_own_webscraper2.py', wdir='C:/Users/Morten/Google Drev/SpyderProject/test')

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/sentdex_Test_comp_screener_own_webscraper2.py", line 60, in <module>
    graphData(stock)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/sentdex_Test_comp_screener_own_webscraper2.py", line 56, in graphData
    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 845, in loadtxt
    converters[i] = conv

IndexError: list assignment index out of range

感谢您 10 月 12 日的意见。2015 戴夫斯·巴姆塞，

但是，正如您所说，我不确定在哪里插入 .join ...

您能否复制此片段并发布您的（编辑的）提案。我需要看到光明！这就是我在 10 月 12 日之前的所有编辑之后所拥有的。

import urllib.request, urllib.error, urllib.parse
import numpy as np
import matplotlib.dates as mdates

stocklist = ['GOOG']


def graphData(stock, MA1, MA2):
    #######################################
    #######################################
    '''
        Use this to dynamically pull a stock from Quandl:
    '''
    print('Currently Pulling',stock)

    netIncomeAr = []

    endLink = 'order=asc'   # order=asc&

    netIncome = str(urllib.request.urlopen('https://www.quandl.com/api/v3/datasets/RAYMOND/'+stock.upper()+'_NET_INCOME_A.csv?'+endLink).read())[2:-1]
    # convert to string, remove leading "b'" and trailing "'" characters.
    # netIncome = 'head\\ndata\\ndata\\n...'


    splitNI = netIncome.split('\\')[1:-1]
    # data segments still have leading 'n' character.
    # the [1:-1] is more pythonic and releases memory.
    for i in range (len(splitNI)):
        splitNI[i] = splitNI[i][1:]
    # data segments are now converted.

    print('Net Income:')
    for eachNI in splitNI:
        print(eachNI)
        netIncomeAr.append(eachNI)


    incomeDate, income = np.loadtxt(netIncomeAr, delimiter=',',unpack=True,
                                    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

for stock in stocklist:
    graphData(stock,25,50)

今天（13-10-2015）来自 Davse Bamse 的输入，我收到以下错误：

Currently Pulling GOOG
Net Income:
2009-12-31,6520448000.0
2010-12-31,8505000000.0
2011-12-31,9737000000.0
2012-12-31,10737000000.0
2013-12-31,12920000000.0
Traceback (most recent call last):

  File "<ipython-input-13-5ce0b8405254>", line 1, in <module>
    runfile('C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py', wdir='C:/Users/Morten/Google Drev/SpyderProject/test')

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 54, in <module>
    graphData(stock,25,50)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 51, in graphData
    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 740, in loadtxt
    fh = iter(open(fname))

OSError: [Errno 22] Invalid argument: '2009-12-31,6520448000.0\n2010-12-31,8505000000.0\n2011-12-31,9737000000.0\n2012-12-31,10737000000.0\n2013-12-31,12920000000.0'

Davse Bamse 建议我像这样使用 io.StringIO：

incomeDate, income = StringIO(np.loadtxt('\n'.join(netIncomeAr), delimiter=',',unpack=True,
                                converters={ 0: mdates.strpdate2num('%Y-%m-%d')}))

但这给了我和以前一样的错误......有什么想法吗？？？

将转换器线更改为：

incomeDate, income = np.loadtxt(StringIO('\n'.join(netIncomeAr)), delimiter=',',unpack=True,
                                converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

给出以下 Stacktrace：

Currently Pulling GOOG
Net Income:
2009-12-31,6520448000.0
2010-12-31,8505000000.0
2011-12-31,9737000000.0
2012-12-31,10737000000.0
2013-12-31,12920000000.0
Traceback (most recent call last):

  File "<ipython-input-26-5ce0b8405254>", line 1, in <module>
    runfile('C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py', wdir='C:/Users/Morten/Google Drev/SpyderProject/test')

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 60, in <module>
    graphData(stock,25,50)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 57, in graphData
    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 860, in loadtxt
    items = [conv(val) for (conv, val) in zip(converters, vals)]

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 860, in <listcomp>
    items = [conv(val) for (conv, val) in zip(converters, vals)]

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\matplotlib\dates.py", line 261, in __call__
    return date2num(datetime.datetime(*time.strptime(s, self.fmt)[:6]))

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\_strptime.py", line 494, in _strptime_time
    tt = _strptime(data_string, format)[0]

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\_strptime.py", line 306, in _strptime
    raise TypeError(msg.format(index, type(arg)))

TypeError: strptime() argument 0 must be str, not <class 'bytes'>

而不是 Numpy 的（我在 np 1.9.2）loadtxt 我找到了另一种方法 np.genfromtxt，显然可以做到这一点在这个解决方案numpy.loadtxt 不读取文件中描述的复数。

所以改用这个转换器线

incomeDate, income = np.genfromtxt('\n'.join(netIncomeAr), delimiter=',',unpack=True,
                                converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

输出

Currently Pulling GOOG
Net Income:
2009-12-31,6520448000.0
2010-12-31,8505000000.0
2011-12-31,9737000000.0
2012-12-31,10737000000.0
2013-12-31,12920000000.0
Traceback (most recent call last):

  File "<ipython-input-10-5ce0b8405254>", line 1, in <module>
    runfile('C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py', wdir='C:/Users/Morten/Google Drev/SpyderProject/test')

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 682, in runfile
    execfile(filename, namespace)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 50, in <module>
    graphData(stock,25,50)

  File "C:/Users/Morten/Google Drev/SpyderProject/test/Test_sentdex_comp_screener_own_webscraper2.py", line 47, in graphData
    converters={ 0: mdates.strpdate2num('%Y-%m-%d')})

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\npyio.py", line 1366, in genfromtxt
    fhd = iter(np.lib._datasource.open(fname, 'rb'))

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\_datasource.py", line 151, in open
    return ds.open(path, mode)

  File "C:\Program Files\WinPython-64bit-3.3.5.7\python-3.3.5.amd64\lib\site-packages\numpy\lib\_datasource.py", line 501, in open
    raise IOError("%s not found." % path)

OSError: 2009-12-31,6520448000.0
2010-12-31,8505000000.0
2011-12-31,9737000000.0
2012-12-31,10737000000.0
2013-12-31,12920000000.0 not found.

我不知道这件事是更好还是更糟......

4

2 回答 2

0

正如 Roland 指出的那样，问题在于返回的是字节数组而不是字符串。

然而，代码应如下所示：

netIncomeBytes = urllib.request.urlopen('https://www.quandl.com/api/v3/datasets/RAYMOND/'+stock.upper()+'_NET_INCOME_A.csv?'+endLink).read()
netIncome = netIncomeBytes.decode("utf-8")

这会将字节数组转换为 utf-8 中的字符串。

于 2015-10-05T10:41:12.327 回答
0

在 Python 3.x 中， urllib.request.urlopen(...).read() 函数如果成功，则返回一个ByteArray -而不是一个字符串对象。

将 ByteArray 转换为 String 的解决方案如下：

...
netIncome = str(urllib.request.urlopen('https://www.quandl.com/api/v3/datasets/RAYMOND/'+stock.upper()+'_NET_INCOME_A.csv?'+endLink).read())[2:-1]
# convert to string, remove leading "b'" and trailing "'" characters.
# netIncome = 'head\\ndata\\ndata\\n...'
...

splitNI = netIncome.split('\\')[1:-1]
# data segments still have leading 'n' character.
# the [1:-1] is more pythonic and releases memory.
for i in range (len(splitNI)):
    splitNI[i] = splitNI[i][1:]
# data segments are now converted.

print('Net Income:')
for eachNI in splitNI:
    print(eachNI)
    netIncomeAr.append(eachNI)
于 2015-10-02T08:14:38.737 回答