2

出于某种原因，我只能从这段代码中获得 100 条推文。根据 Twitter API，我相信我应该得到 1500。

我在这里做错了什么？

具体来说，有问题的问题是：

twiturl = "http://search.twitter.com/search.json?q=" + urlinfo + "&rpp=99&page=15" + "&since_id=" + str(tweetdate)

for x in arg1:
                        urlinfo = x[2]
                        idnum = int(x[1])
                        name = x[0]
                        twiturl = "http://search.twitter.com/search.json?q=" + urlinfo + "&rpp=99&page=15" + "&since_id=" + str(tweetdate)
                        response = urllib2.urlopen(twiturl)
                        twitseek = simplejson.load(response)
                        twitsearch = twitseek['results']
                        tweets = [x['text'] for x in twitsearch]
                        tweetlist = [tweets, name]
                        namelist.append(tweetlist)

应该在 x[2] 中的项目只是一个单词或短语，如“我是”或“我感觉”更改为 url 友好编码

4

2 回答 2

3

Twitter 搜索 API状态的文档：

rpp（可选）：每页返回的推文数量，最多 100 条。

和

page（可选）：要返回的页码（从 1 开始），最多大约 1500 个结果（基于 rpp * page）。

因此，您应该发出多个请求，每个请求都有一个不同的页码，每个请求最多 100 条推文：

import urllib, json

twiturl = "http://search.twitter.com/search.json?q=%s&rpp=99&page=%d"

def getmanytweets(topic):
    'Return a list of upto 1500 tweets'
    results = []
    for page in range(1, 16):
        u = urllib.urlopen(twiturl % (topic, page))
        data = u.read()
        u.close()
        t = json.loads(data)
        results += t['results']
    return results

if __name__ == '__main__':
    import pprint
    pprint.pprint(getmanytweets('obama'))
于 2012-12-30T02:17:16.690 回答
2

单页结果中返回的最大结果数为 100。为了获得所有结果，您需要使用next_page响应中包含的 URL 来“翻页”它们（有关文档，请参见此处）。然后，您可以遍历响应，调用next_page每个响应的参数，直到参数不再存在（表明您已收集所有结果）。

import json
import urllib
import urllib2


# General query stub
url_stub = 'http://search.twitter.com/search.json'

# Parameters to pass
params = {
    'q': 'tennis',
    'rpp': 100,
    'result_type': 'mixed'
    }

# Variable to store our results
results = []

# Outside of our loop, we pull the first page of results
# The '?' is included in the 'next_page' parameter we receive
# later, so here we manually add it
resp = urllib2.urlopen('{0}?{1}'.format(url_stub, urllib.urlencode(params)))
contents = json.loads(resp.read())
results.extend(contents['results'])

# Now we loop until there is either no longer a 'next_page' variable
# or until we max out our number of results
while 'next_page' in contents:

  # Print some random information
  print 'Page {0}: {1} results'.format(
      contents['page'], len(contents['results']))

  # Capture the HTTPError that will appear once the results have maxed
  try:
    resp = urllib2.urlopen(url_stub + contents['next_page'])
  except urllib2.HTTPError:
    print 'No mas'
    break

  # Load our new contents
  contents = json.loads(resp.read())

  # Extend our results
  results.extend(contents['results'])

# Print out how many results we received
print len(results)

输出：

Page 1: 103 results
Page 2: 99 results
Page 3: 100 results
Page 4: 100 results
Page 5: 100 results
Page 6: 100 results
Page 7: 100 results
Page 8: 99 results
Page 9: 98 results
Page 10: 95 results
Page 11: 100 results
Page 12: 99 results
Page 13: 99 results
Page 14: 100 results
Page 15: 100 results
No mas
1492
于 2012-12-30T01:47:50.500 回答