0

我正在建立一个项目。我现在正在处理的部分应该执行以下操作：

  1. 使用 ID 和密码登录平台（完成）
  2. 获取数据数组（暂时硬编码）
  3. 使用数组的每个元素执行 API 调用，获取每个元素的结果并将其放入另一个数组
  4. 简而言之，通过传递一个 id 数组作为输入，我应该得到一个结果数组作为输出

为了进一步简化问题，我决定使用一个 id 并将其传递 1000 次，而不是 1000 个不同值的数组。

限制

我可以每分钟调用 API 300 次，否则我会得到著名的“429 问题”->

我的应用程序不使用 express。我正在寻找替代品并遇到了瓶颈包。它完全符合我的需要，并且应该限制 API 调用以及在失败或事件管理的情况下执行重试（请参阅上面链接中的文档）。

这是我到目前为止所拥有的：

const BottleNeck = require('bottleneck');
const logger = require('./functions/logger');
/**
 * this reservoir is allowed to execute 300 requests/ minute
 */
const limiter = new BottleNeck({
    reservoir: 300, // 300 requests per minute
    reservoirRefreshAmount: 249,
    reservoirRefreshInterval: 60 * 1000, // must be divisible by 250

    trackDoneStatus: true,

    minTime: 300//5 requests per second
});
limiter.on("debug", function (message, data) {
    // Useful to figure out what the limiter is doing in real time
    // and to help debug your application
});
limiter.on("depleted", function (empty) {
    logger.info(`BottleNeck() - reservoir is empty. Waiting... ${limiter.options}`);
    // This will be called every time the reservoir drops to 0.
    // The `empty` (boolean) argument indicates whether `limiter.empty()` is currently true.
});
limiter.on("dropped", function (dropped) {
    // This will be called when a strategy was triggered.
    // The dropped request is passed to this event listener.
});
limiter.on("empty", function () {
    // This will be called when `limiter.empty()` becomes true.
});
limiter.on("error", function (error) {
    /* handle errors here */
    logger.error(`Limiter : fetched error ${error.message}`);
});

// Listen to the "failed" event
limiter.on("failed", async (error, jobInfo) => {
    const id = jobInfo.options.id;
    console.warn(`Job ${id} failed: ${error}`);

    if (jobInfo.retryCount === 0) { // Here we only retry once
        console.log(`Retrying job ${id} in 25ms!`);
        return 25;
    }
});

limiter.on("idle", function () {
    // This will be called when `limiter.empty()` is `true` and `limiter.running()` is `0`.
});
limiter.on("queued", function (info) {
    // This event is triggered when a job transitions from one Lifecycle stage to another
});
// Listen to the "retry" event
limiter.on("retry", (error, jobInfo) => console.log(`Now retrying ${jobInfo.options.id}`));

module.exports = { bottleNeck: limiter };

（假设 logger 是使用 logs4js 或任何其他方式定义的。

主要代码：

const { logger } = require("./functions/logger");
const { moment } = require("moment");
const { toCloudLogin, youOnlyHadOneJob } = require("./functions/toCloud");
const { bottleNeck } = require("./apiBottleNeck");



const go = async () => {
    try {

        logger.info(`hello`);
        await toCloudLogin();


        logger.info(`starting the bottleneck`);
        let counter = 0;
        for (let x = 0; x < 310; x++) {                           //calling same job through bottleneck many times
            counter+=1;
            let reservoir = await bottleNeck.currentReservoir(); // returns integer of the current 
                                                                 // reservoir api calls left
            const result = await bottleNeck.schedule(async () => youOnlyHadOneJob(reservoir));

        }
        logger.info(`finished the bottleneck` + counter)

    } catch (e) {
        console.log(e);
        logger.error(`error: ${e.message} stack :${e.stack}`);
    }
}
go();

以下是我的问题：

  • 如果bottleNeck应该过滤API调用的数量，为什么我仍然得到最大调用堆栈大小超过知道我启动300存储（实际上是301和0）和300充电

  • 我应该如何实现所有这些事件？（例如耗尽：

limiter.on("depleted", function (empty) {
    logger.info(`BottleNeck() - reservoir is empty. Waiting... ${limiter.options}`);
    // This will be called every time the reservoir drops to 0.
    // The `empty` (boolean) argument indicates whether `limiter.empty()` is currently true.
});

）。在我的主要功能中，以便在水库为空时向我发送任何内容。我已经阅读了事件发射器，我认为我应该在某个地方limiter.emit(耗尽)，但我不明白如何正确整合它。

4

0 回答 0