php - Split a big txt file with php

Question

I have a big txt file (84mb) with over 1 million lines.

I want to split it in seperate 50k lines files. How can i do it? I searched on-line but I found nothing about it.

Answer 1

This is a modified version of my script at : Read through huge text files and store each line in database

Usage

set_time_limit(0);

// Split by Byte
splitData("b.php", __DIR__ . "/test", 1024 * 50); //Split 50Kb

// Split By line
splitLine("b.php", __DIR__ . "/test", 50000);

Functions

function splitData($filename, $destination, $chunkSize) {
    $pathInfo = pathinfo($filename);
    $handle = fopen($filename, 'rb');
    $counter = 0;
    if ($handle === false) {
        return false;
    }
    while ( ! feof($handle) ) {
        $counter ++;
        $filePart = $destination . DIRECTORY_SEPARATOR . $pathInfo['filename'] . "_" . $counter . "." . $pathInfo['extension'];
        touch($filePart);
        file_put_contents($filePart, fread($handle, $chunkSize));
    }
    $status = fclose($handle);
    return $status;
}

function splitLine($filename, $destination, $lineSize) {
    $pathInfo = pathinfo($filename);
    $handle = fopen($filename, 'rb');
    $counter = 0;
    $splitCount = 0;
    if ($handle === false) {
        return false;
    }

    $content = "";
    while ( ($buffer = fgets($handle, 4096)) !== false ) {
        $content .= $buffer;
        $counter ++;

        if ($counter >= $lineSize) {
            $splitCount ++;
            $filePart = $destination . DIRECTORY_SEPARATOR . $pathInfo['filename'] . "_" . $splitCount . "." . $pathInfo['extension'];
            touch($filePart);
            file_put_contents($filePart, $content);
            $content = "";
            $counter = 0;
        }
    }
    $status = fclose($handle);
    return $status;
}

Answer 2

<?php 
$handle = @fopen("/tmp/inputfile.txt", "r");
$maxLines = 50;
if ($handle) {
    $counter = 1;
    $fileCount = 1;
    $data = array();
    while (($buffer = fgets($handle, 4096)) !== false) {
        $data[] = $buffer;
        if(count($data) % $maxLines == 0) {
            file_put_contents("filename{$fileCount}.txt", implode("\n\r", $data));
            $data = array();
            $fileCount++;
        }
        $counter++;
    }
    if (!feof($handle)) {
        echo "Error: unexpected fgets() fail\n";
    }
    fclose($handle);
}  
?>

像这样的东西应该可以工作，尽管我永远不会推荐它，这不是解决问题的好方法。