1

I have a PHP script file that loads a series of CSV files into a MYSQL database.

when I run the script from the command line all going well but when I run it from a browser it quits in the middle (after around 65000 records) and 4 files.

<?

if (isset($argv)) 
{
    $_GET['load_date'] = $argv[1];
}

LoadFile('file1.csv');
LoadFile('file2.csv');
LoadFile('file3.csv');
LoadFile('file4.csv');
LoadFile('file5.csv');
LoadFile('file6.csv');
LoadFile('file7.csv');

function LoadFile($File_Name) 
{

    //global $serverinfo, $username, $password, $database, $dir_path, $dir_env;
    include("datacon.inc.php");

    mysql_connect($serverinfo, $username, $password);
    @mysql_select_db($database) or die("Unable to select database");

    //check if the file is existed
    if (file_exists($File_Name) == FALSE) {
        ECHO "<FONT COLOR=\"red\"> <b> " . $File_Name . " wasn't found </b> </FONT> <br>";
        return;
    } else {
        ECHO $File_Name . " was found, start loading...<br>";
    }

    //Import uploaded file to Database
    $file_handle = fopen($File_Name, "r");

    while (($line_of_data = fgetcsv($file_handle, 0, ",", "\"", "\r\n")) !== FALSE) {


        $line_import_query = "INSERT STATEMENT...";

        //echo $line_import_query . "<BR><br>"; 
        mysql_query($line_import_query) or die(mysql_error());
    }

    $Count_records_query =
            "SELECT COUNT(*) AS COUNTS FROM `" . $database . "`.`TBL_TABLE`";

    $Count_records_query_result = mysql_query($Count_records_query);
    $Count_records = mysql_result($Count_records_query_result, 0, "COUNTS");

    if ($Count_records <= 0) {
        mysql_close();
        echo "No records were loaded on " . $File_Name . ", somthing is wrong, check the file location/structure <br>";
    } else {
        echo "<FONT COLOR=\"red\"> <b> " . $File_Name . " were inserted </b> </FONT> <br>";
        # Disconnect from the database.
    }

    fclose($File_Name, "r");
    // close the connection
    mysql_close();
    return;
}

echo "Disconnected from database successfully! <br><br>
           <input type=\"Button\" value=\"Ok, Finished! Back\" onclick=\"history.back()\">";
?>

What's wrong? How can I debug this?

4

2 回答 2

3

This is a timeout problem due to the fact that you are using a IIS server. The maximum execution time is 30 seconds for an entire process and setting max_execution_time in php.ini or set_time_limit() in the php code are ignored.

Under an Apache server, the 30 seconds is only for script execution; the time used by IO (file read, SQL request, ...) is deducted.

But it is not a good idea to change max_execution_time in php.ini because this affects all the php threads and may overload the server due to many requests that remain opened until timeout expiration.

Exceptionaly you might change locally this setting with set_time_limit(); in your php code

Another solution is to divide the process using a page refresh.

In this sample the file is "LoadCSV.php" (used in header command) 

If (isset($_SESSION['ProcessId'])){  // process Phase    
  $P=$_SESSION['ProcessId'];
  LoadFile($_SESSION['ProcessArray'][$P]);  //All echo must be removed from LoadFile !!! 
  $P+=1;
  if ($P<(count($_SESSION['ProcessArray'])-1)){
    header('location: LoadCSV.php?Action=ImportFiles');
  }
  else{  // end Phase
     unset($_SESSION['ProcessArray']);
     unset($_SESSION['ProcessId']);

     echo 'Finished';

  }
}
else{  // Init Phase
  $LoadArray=array(
    'file1.csv',
    'file2.csv',
    'file3.csv',
    'file4.csv',
    'file5.csv',
    'file6.csv',
    'file7.csv'
  );
  $_SESSION['ProcessArray']=$LoadArray;
  $_SESSION['ProcessId']=0;
  header('location: LoadCSV.php?Action=ImportFiles');
}
于 2013-06-16T13:23:40.483 回答
0

try ini_set('max_execution_time', 0); to set the php script to run until it has finished.

于 2013-06-16T12:31:38.053 回答