0

我一直在从事一个研究项目，涉及从 twitter 获取朋友和关注者信息。我编写的代码适用于像我这样拥有 1k 关注者或朋友的示例帐户。但我在为拥有 1220 万粉丝的唐纳德特朗普账户运行它时遇到了麻烦。在我的程序中查询超出了某个时间速率限制后，我等待 15 分钟，然后再次请求相同的。此过程适用于大约 100 万用户信息，但之后会卡住，或者有时会出现 URL 错误。请提出在处理如此大的数据时可能出现的问题以及解决方案。请在下面找到相同的代码片段 -

def get_info_by_screen_name(t, screen_names):

    sn_to_info = {}
    while len(screen_names) > 0:

        # Process 100 ids at a time...

        screen_names_str = ','.join([str(sn) for sn in screen_names[:100]])
        screen_names = screen_names[100:]

        response = make_twitter_request(t,
                                        getattr(getattr(t, "users"), "lookup"),
                                        screen_name=screen_names_str)

        if response is None:
            break

        if type(response) is dict:  # Handle Twitter API quirk
            response = [response]

        for user_info in response:
            sn_to_info[user_info['screen_name']] = user_info

    return sn_to_info


    t = oauth_login()

    friends_ids, followers_ids = get_friends_followers_ids(t,
                                                           screen_name="realDonaldTrump",
                                                           friends_limit=100,
                                                           followers_limit=100)

    # Basic usage...


    print friends_ids
    print followers_ids
    # Basic usage...
    info = {}
    data_test = {}
    info.update(get_info_by_id(t, friends_ids))
4

0 回答 0