3

我正在使用 asyncio 和 pyppeteer 来测试抓取网站。目前我有：

browser = await launch(
        args=[f'--proxy-server={proxyUrl}'],
        headless=True,
        autoClose=False
    )

启动浏览器。我在autoClose=False运行它时使用它，因为True我留下了挂起的 Chromium 进程。

在与许多网络错误作斗争之后，我发现这是最好的方法，并且挂起进程是最好的方法：

def close_browser(browser):
    browser.process.terminate()

但是，有时该操作不会引发任何错误，或者最多 18 个错误。所有的错误是：

Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.

这是我循环代码 10 次时的输出：

Completed  Number 4

Completed  Secondary

Completed  Primary

Completed  Number 5
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.

Completed  Tertiary
Done
Running all tasks took  8.363972902297974
All tasks successfully ran.

Completed  Secondary

Completed  Tertiary

Completed  Number 4

Completed  Primary

Completed  Number 5
Done
Running all tasks took  9.243869304656982
All tasks successfully ran.

Completed  Tertiary

Completed  Primary

Completed  Number 4

Completed  Number 5

Completed  Secondary
Done
Running all tasks took  8.441104650497437
All tasks successfully ran.

Completed  Number 5

Completed  Primary

Completed  Secondary

Completed  Number 4

Completed  Tertiary
Done
Running all tasks took  8.054714918136597
All tasks successfully ran.

Completed  Number 5

Completed  Number 4

Completed  Secondary

Completed  Tertiary

Completed  Primary
Done
Running all tasks took  8.639416217803955
All tasks successfully ran.

Completed  Number 5

Completed  Secondary

Completed  Number 4

Completed  Tertiary

Completed  Primary
Done
Running all tasks took  8.757359027862549
All tasks successfully ran.

Completed  Tertiary

Completed  Number 5

Completed  Primary

Completed  Number 4

Completed  Secondary
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.
Done
Running all tasks took  8.711585521697998
All tasks successfully ran.

Completed  Number 4

Completed  Tertiary

Completed  Secondary

Completed  Number 5
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.
Future exception was never retrieved
future: <Future finished exception=NetworkError('Protocol error Target.sendMessageToTarget: Target closed.')>
pyppeteer.errors.NetworkError: Protocol error Target.sendMessageToTarget: Target closed.

Completed  Primary
Done
Running all tasks took  8.28357720375061
All tasks successfully ran.

Completed  Tertiary

Completed  Secondary

Completed  Number 5

Completed  Primary

Completed  Number 4
Done
Running all tasks took  8.737911939620972
All tasks successfully ran.

Completed  Secondary

Completed  Number 5

Completed  Number 4

Completed  Primary

Completed  Tertiary
Done
Running all tasks took  8.866266965866089
All tasks successfully ran.
4

0 回答 0