1

我写了一个工匠命令来从数据库中导出练习,到可以在像moodle这样的电子学习系统中使用的独立包,......

这是大量的练习,一段时间后记忆就会耗尽。

我尝试取消设置变量,激活垃圾收集器,禁用查询日志,并进行了一些分析,但直到现在都没有成功

我在下面附上了我的脚本,我处理的每个练习,内存使用量加起来 300k 任何想法我能做什么?

use Illuminate\Console\Command;
use Illuminate\Filesystem\Filesystem;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputArgument;

set_time_limit(0);

class ExerciseExportCommand extends Command {

    /**
     * The console command name.
     *
     * @var string
     */
    protected $name = 'exercises:export';

    /**
     * The console command description.
     *
     * @var string
     */
    protected $description = 'Export all exercises of a method or a specific exercise.';

    /**
     * Create a new command instance.
     *
     * @return void
     */
    public function __construct()
    {
        parent::__construct();
    }

    /**
     * Execute the console command.
     *
     * @return void
     */
    public function fire()
    {
        try {

            DB::disableQueryLog();

            ini_set('memory_limit','1024M');

            $this->info('Initial: ' . number_format(memory_get_usage(), 0, '.', ',') . " bytes\n");

            $base = base_path() . "/export/";
            $playerPath = base_path() . "/public/preview/dist/";
            $uploadsPath = base_path() . "/public/uploads/";

            $methodId = $this->option('method');
            $categoryId = $this->option('category');
            $exerciseId = $this->option('exercise');

            $this->info("Swing baby...");
            $this->info("Let's export some exercises, shall we ?");


            //we make an array which holds all the exercises we have to export
            $exercises = array();

            if($methodId === NULL && $categoryId === NULL && $exerciseId === NULL){
                //we are here now anyways, let's do all exercises at once...
                $this->comment("Nothing specified, let's do all exercises ");
                $exercises2 = Exercise::all();
                foreach ($exercises2 as $exercise){
                    array_push($exercises, $exercise->id);
                }
                unset($exercises2);

            }

            //get all exercises for given methodId
            if($methodId !== NULL){
                $method = Method::with('categories.exercises')->find($methodId);
                if($method == NULL) break;

                $this->comment("We are ready to roll method " . $method->code);
                foreach($method->categories as $category){
                    foreach($category->exercises as $exercise->id){
                        array_push($exercises, $exercise);
                    }
                }
                unset($method);
            }

            //get all exercises for given categoryId
            if($categoryId !== NULL){
                $category = Category::with('exercises')->find($categoryId);
                if($category == NULL) break;

                $this->comment("We are ready to roll category " . $category->name_prefix . " " . $category->name);
                foreach($category->exercises as $exercise->id){
                    array_push($exercises, $exercise);
                }
                unset($category);
            }


            if($exerciseId != null){
                $exercise = Exercise::find($exerciseId);
                if($exercise != NULL) {
                    array_push($exercises, $exercise->id);
                    $this->comment("Exercise added for export: " . $exercise->name_prefix . " " . $exercise->name);
                } else {

                }
                unset($exercise);
            }

            if(empty($exercises)){
                $this->error("No exercises could be found for given method/exercise");
                exit();
            } else {
                $this->comment("Currently counting " . count($exercises) . " exerises to export");
            }

            $fs = new Filesystem();

            //loop the exercises and publish like a charm
            foreach($exercises as $exerciseId){
                $exercise = Exercise::find($exerciseId);
                //determine destination
                $path = $base . $exercise->getPath();
                $this->comment("starting exercise " . $exercise->id);

                //check if path exists, if it does, wipe it out
                if($fs->exists($path)){
                    $fs->deleteDirectory($path, true);
                    $this->comment("wiped out " . $path);
                }

                //copy player files
                //echo "copying " . $path . "<br />";
                $fs->copyDirectory($playerPath, $path);

                $fs->cleanDirectory($path."styles/skins");

                    //copy only necesary skin files to save disk space
                    $method = $exercise->method();

                    if($fs->exists($playerPath."styles/skins/".$method->code)){
                        $fs->copyDirectory($playerPath."styles/skins/".$method->code, $path."styles/skins/".$method->code);
                    } elseif($method->code == "kameleonspelling" || $method->code == "kameleontaalbeschouwing"){
                        $fs->copyDirectory($playerPath."styles/skins/kameleon", $path."styles/skins/kameleon");
                    }

                    if($fs->exists($playerPath."styles/skins/".$method->code.".css")){
                        $fs->copy($playerPath."styles/skins/".$method->code.".css", $path."styles/skins/".$method->code.".css");
                    }

                $this->comment("copied player files to " . $path);

                //copy resources
                //echo "copying resources " . $path . "<br />";
                $fs->copyDirectory($uploadsPath . $exercise->id . "/", $path);
                $this->comment("copied resources to " . $path);

                //copy slide resources
                $slides = Slide::where('exerciseID',"=",$exercise->id)->get();
                mkdir($path."slides/");

                foreach ($slides as $slide) {
                    $image = $slide->image()->first();
                    if($image != NULL){
                        $this->info($uploadsPath."slides/".$image->resourceUri);
                        $this->info($path."slides/".$image->resourceUri);
                        $fs->copy($uploadsPath."slides/".$image->resourceUri, $path."slides/".$image->resourceUri);
                    }
                    unset($image);
                }
                $this->comment("copied slide resources to " . $path);

                //save xml file
                $content = Exercise::getXmlContent($exercise->id);
                $fs->put($path . "exercise.xml", View::make('xml', $content));
                $this->comment("saved xml to " . $path);

                $this->info("finished exercise " . $exercise->id);

                unset($method);
                unset($content);
                unset($slides);
                gc_collect_cycles();
                $this->info('Peak: ' . number_format(memory_get_peak_usage(), 0, '.', ',') . " bytes\n");
                $this->info('End: ' . number_format(memory_get_usage(), 0, '.', ',') . " bytes\n");
            }

            $this->info("Awesome Possum => finished all exercises ");
            $this->info('Peak: ' . number_format(memory_get_peak_usage(), 0, '.', ',') . " bytes\n");
            $this->info('End: ' . number_format(memory_get_usage(), 0, '.', ',') . " bytes\n");

        } catch(Exception $e){
            $this->error($e->getMessage());
            $this->comment($e->getTraceAsString());
        }
    }

    /**
     * Get the console command arguments.
     *
     * @return array
     */
    protected function getArguments()
    {
        return array(
            //array('example', InputArgument::REQUIRED, 'An example argument.'),
        );
    }

    /**
     * Get the console command options.
     *
     * @return array
     */
    protected function getOptions()
    {
        return array(
            array('method', null, InputOption::VALUE_OPTIONAL, 'The id of a method  for which all the exercises to export.', null),
            array('category', null, InputOption::VALUE_OPTIONAL, 'The id of a category for which all the exercises to export.', null),
            array('exercise', null, InputOption::VALUE_OPTIONAL, 'The id of an exercise to export.', null),
        );
    }

}

这是我的 xdebug trace 命令的转储,其中包含 20 个最消耗内存的语句:

Showing the 20 most costly calls sorted by 'memory-own'.

                                                                       Inclusive        Own
function                                                       #calls  time     memory  time     memory
-------------------------------------------------------------------------------------------------------
debug_backtrace                                                   646  0.0420 20353496  0.0420 20353496
Composer\Autoload\ClassLoader->loadClass                          259  0.1911 17556224  0.1139 13953824
PDOStatement->execute                                             743  0.1184 13729408  0.1184 13729408
array_merge                                                      4051  0.1282  3894816  0.1282  3894816
Illuminate\Database\Eloquent\Model->newInstance                  1534  0.4715  3806344  0.0791  3732712
PDOStatement->fetchAll                                            742  0.0323  2364264  0.0323  2364264
Illuminate\Database\Eloquent\Model->newBaseQueryBuilder           738  0.6625  2177352  0.0657  1688968
explode                                                          3396  0.1026  1296960  0.1026  1296960
Illuminate\Database\Eloquent\Model->newFromBuilder               1534  0.6883  5139552  0.0944  1259576
str_replace                                                     10254  0.3176  1228824  0.3176  1228824
compact                                                           920  0.0339  1181384  0.0339  1181384
PDO->prepare                                                      743  0.1403   816488  0.1403   816488
sprintf                                                          2381  0.0741   802968  0.0741   802968
implode                                                          5586  0.1722   536688  0.1722   536688
array_map                                                         864  0.3164   588512  0.0386   477088
get_class_methods                                                  15  0.0059   472296  0.0059   472296
Illuminate\Database\Eloquent\Model->newQuery                      738  0.9783  3044352  0.0656   448488
include                                                           263  6.7525  5732672  0.0468   410416
call_user_func_array                                             1585  0.5734  3937936  0.0659   357056
SplFileInfo->getPathname                                         2724  0.0847   344768  0.0847   344768
4

2 回答 2

4

事实证明,DB::disableQueryLog();它毕竟修复了它。!!

起初我认为它没有帮助,因为内存不断增加,我每次都手动取消我的脚本。现在,在使用 Memtrack 进行调试时,我一直在运行我的命令,但过了一会儿我注意到内存使用量停滞不前。

我猜垃圾收集器在决定需要它之前不会清理内存?

于 2013-10-18T11:39:53.683 回答
0

您的服务器 memory_limit 是多少?

一种解决方案可能是将命令拆分为更小的命令,并相应地运行它们。也许在一个完成后自动运行一个新命令?

我还建议使用某种队列提供程序。通过这种方式,您可以将工作负载拆分为更长的时间段。此外,工作负载根本不分布在您的服务器上。

Laravel 内置支持:Pheanstalk、Amazon SQS 和 IronMQ。

这是文档链接:Laravel 队列文档

于 2013-10-18T11:22:10.267 回答