0

i got a function in PHP to read table from ODBC (to IBM AS400) and write it to a text file on daily basis. it works fine until it reach more than 1GB++. Then it just stop to some rows and didn't write completely.

function write_data_to_txt($table_new, $query)
{
    global $path_data;
    global $odbc_db, $date2;

    if(!($odbc_rs = odbc_exec($odbc_db,$query))) die("Error executing query $query");
    $num_cols = odbc_num_fields($odbc_rs);

    $path_folder = $path_data.$table_new."/";
    if (!file_exists($path_folder)) mkdir ($path_folder,0777);

    $filename1 = $path_folder. $table_new. "_" . $date2 . ".txt";
    $comma = "|";
    $newline = chr(13).chr(10);

    $handle = fopen($filename1, "w+");

    if (is_writable($filename1)) {
      $ctr=0; 
        while(odbc_fetch_row($odbc_rs))
        {
        //function for writing all field
        //  for($i=1; $i<=$num_cols; $i++)
        //  {
            //  $data = odbc_result($odbc_rs, $i);
            //  if (!fwrite($handle, $data) || !fwrite($handle, $comma)) {
                //  print "Cannot write to file ($filename1)";
                //  exit;
            //  }
            //}
        //end of function writing all field
            $data = odbc_result($odbc_rs, 1);
            fwrite($handle,$ctr.$comma.$data.$newline);
            $ctr++;
        }
        echo "Write Success. Row = $ctr <br><br>";
    }
    else
    {
        echo "Write Failed<br><br>";
    }
    fclose($handle);
}

no errors, just success message but it should be 3,690,498 rows (and still increase) but i just got roughly 3,670,009 rows

My query is ordinary select like : 

select field1 , field2, field3 , field4, fieldetc from table1

What i try and what i assume :

  1. I think it was fwrite limitation so i try not to write all field (just write $ctr and 1st record) but it still stuck in same row.. so i assume its not about fwrite exceed limit..
  2. I try to reduce field i select and it can works completely!! so i assume it have some limitation on odbc.
  3. I try to use same odbc datasource with SQL Server and try to select all field and it give me complete rows. So i assume its not odbc limitation.
  4. Even i try on 64 bits machine but it even worse, it just return roughly 3,145,812 rows.. So i assume it's not about 32/64 bit infrastructure.
  5. I try to increase memory_limit in php ini to 1024mb but it didnt work also..

Is there anyone know if i need to set something in my PHP to odbc connection??

4

0 回答 0