0

我正在使用 Laravel 5.7 制作网站爬虫。队列在 Redis 驱动程序上,我正在使用 laravel Horizo​​n 来运行该作业。我只想同时运行或排队一个作业,但是它有时会复制一个作业并最终同时运行同一作业的多个实例。有什么方法可以保证一次只运行一个作业?

职位类别

class MyJob implements ShouldQueue

{ 使用 Dispatchable、InteractsWithQueue、Queueable、SerializesModels;

public $timeout = 30000;

public $tries = 1;

public function __construct($jobid = null)
{
    if (!$jobid) {
        $this->ra_jobid = substr(str_shuffle('1234567890abcdefghijklmnopqrstuvwxyz'), 0, 8);
    } else {
        $this->ra_jobid = $jobid;
    }
}

public function handle()
{
    Redis::funnel('RetrieveAucfanData')->limit(1)->then(function () {
        if (Redis::get('stop_job' . $this->ra_jobid)) {
            Redis::del('stop_job' . $this->ra_jobid);
            throw new JobManuallyStoppedException();
        }

        //crawl a website

    }, function () {
        return $this->release(60);
    });
}

public function failed(\Exception $exception)
{
    if ($exception->getMessage() == 'Job Manually Stopped') {
        Log::debug($exception->getMessage());
    } else {
        self::dispatch($this->ra_jobid)
            ->delay(now()->addSeconds(60));
    }

}

}

如果作业失败,我想再次将该作业排队,除非它被手动停止。如果作业达到限制,我还想将作业放回队列。

队列.php

<?php

返回 [

/*
|--------------------------------------------------------------------------
| Default Queue Driver
|--------------------------------------------------------------------------
|
| Laravel's queue API supports an assortment of back-ends via a single
| API, giving you convenient access to each back-end using the same
| syntax for each one. Here you may set the default queue driver.
|
| Supported: "sync", "database", "beanstalkd", "sqs", "redis", "null"
|
*/

'default' => env('QUEUE_DRIVER', 'redis'),

/*
|--------------------------------------------------------------------------
| Queue Connections
|--------------------------------------------------------------------------
|
| Here you may configure the connection information for each server that
| is used by your application. A default configuration has been added
| for each back-end shipped with Laravel. You are free to add more.
|
*/

'connections' => [

    'sync' => [
        'driver' => 'sync',
    ],

    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 30,
    ],

    'beanstalkd' => [
        'driver' => 'beanstalkd',
        'host' => 'localhost',
        'queue' => 'default',
        'retry_after' => 30,
    ],

    'sqs' => [
        'driver' => 'sqs',
        'key' => 'your-public-key',
        'secret' => 'your-secret-key',
        'prefix' => 'https://sqs.us-east-1.amazonaws.com/your-account-id',
        'queue' => 'your-queue-name',
        'region' => 'us-east-1',
    ],

    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => 'default',
        'retry_after' => 300,
        'block_for' => 10,
    ],

],

/*
|--------------------------------------------------------------------------
| Failed Queue Jobs
|--------------------------------------------------------------------------
|
| These options configure the behavior of failed queue job logging so you
| can control which database and table are used to store the jobs that
| have failed. You may change them to any database / table you wish.
|
*/

'failed' => [
    'database' => env('DB_CONNECTION', 'mysql'),
    'table' => 'failed_jobs',
],

];

4

1 回答 1

0

您的超时值大于 retry_after 值,因此两个或多个工作人员正在处理相同的作业。

我在这里添加了文档的快照。

于 2021-01-04T03:49:48.513 回答