2

我有一个附属窗口 CSV，它有近 1 个 lac 行，我想将它保存为节点，我尝试使用批处理 API。

但我仍然收到php超时错误..请帮助

function MODULE_aw_batch(){  
  $operations = array();
  $csv = file_directory_path().'/aw/datafeed_134642.csv';
  $file = fopen($csv, 'r');
  while (($data = fgetcsv($file)) !== FALSE) {
      $operations[] = array('MODULE_aw_op', array($data));
  }
  $batch = array(
    'title' => t('Generating feeds'), // Title to display while running.
    'operations' => $operations,
    'finished' => 'MODULE_aw_finished', // Last function to call.
    'init_message' => t('Importing...it may take 4-5 hours'),
    'progress_message' => t('Processed @current out of @total.'),
    'error_message' => t('Import feeds has encountered an error.'),
  );
  batch_set($batch);
  batch_process('admin/content/node/overview');
}

更新（已解决）

而不是一次读取整个 csv 文件，而是将 csv 拆分为每个进程读取 5 行

4

2 回答 2

2

找到了解决方案

批量申报

function MODULENAME_batch(){
    $operations[] = array('MODULENAME_op', array()); 
    $batch = array(
        'title' => t('Importing events'),
        'operations' => $operations,
        'finished' => 'MODULENAME_finished',
        'init_message' => t('MESSAGE HERE'),
        'progress_message' => 'MESSAGE HERE',
        'error_message' => t('MESSAGE HERE'),
    );
    batch_set($batch);
    batch_process('admin/content/node/overview');// redirect to this page
}

function MODULENAME_aw_op(&$context) {      
    if (!isset($context['sandbox']['offset'])) {
        $context['sandbox']['offset'] = 0;
        $context['sandbox']['records'] = 0;
    }

    $filename =file_directory_path().'/aw/datafeed_134642.csv'; // csv file

    $fp = fopen($filename, 'r');

    if ( $fp === FALSE ) {
    // Failed to open file
        watchdog('MODULENAME', 'Failed to open ' . $filename);
        $context['finished'] = TRUE;
        return;
    }

    $ret = fseek($fp, $context['sandbox']['offset']);

    if ( $ret != 0 ) {
    // Failed to seek
        watchdog('MODULENAME', 'Failed to seek to ' . $context['sandbox']['offset']);
        $context['finished'] = TRUE;
        return;
    }

    $limit = 5;  // Maximum number of rows to process at a time
    $done = FALSE;

    for ( $i = 0; $i < $limit; $i++ ) {
        $line = fgetcsv($fp);
        if ( $line === FALSE ) {
            $done = TRUE;
            // No more records to process
            break;
        }
        else {          
            $context['sandbox']['records']++;
            $context['sandbox']['offset'] = ftell($fp);
            $record = $context['sandbox']['records'];
            // Do something with the data

        }
    }

    $eof = feof($fp);

    if ( $eof )  {
        $context['success'] = TRUE;
    }
    $context['message'] = "processed " . $context['sandbox']['records'] . " records";

    $context['finished'] = ( $eof || $done ) ? 1 : 0;
}
于 2012-07-31T11:05:32.650 回答
1

您是否尝试过 Drupal 提要模块？它非常适合 CSV 导入。 http://drupal.org/project/feeds/

于 2012-06-29T14:05:35.503 回答