1

我需要运行一些 bash 命令，每个命令大约需要 18 小时才能完成。为了使用所有可用的 CPU 内核，我使用以下代码在单独的线程上运行每个命令。然而，线程在工作大约 5 小时后停止做任何事情。我通过检查cpu使用情况知道这一点。当我使用相同的代码只运行一个命令时，它运行良好。此外，我在不同的 bash 终端会话中运行了几个命令，以确保它们真正独立。

public class RunBashCommand implements Runnable{
    private String[] command; // e.g. {"bash", "myScript.sh", "arg1", arg2"} 
    private String fSSubjectFolder; 
    private String subjectId;


    RunBashCommand ( String[] newCommand, String newSubjectFolder, String newSubjectId ) {
        command = newCommand;
        fSSubjectFolder = newSubjectFolder;
        subjectId = newSubjectId;

    }

    public void run ( ){
        runCommand ();
    }

    private void runCommand (){
        Runtime run = Runtime.getRuntime();
        Process p;
        try {
            String line;
            p = run.exec(command);
            BufferedReader buf = new BufferedReader( new InputStreamReader(p.getInputStream() ) );
            PrintWriter pw = new PrintWriter( new File( fSSubjectFolder+"/"+subjectId+"CTPFS.log" ) );
            while ( ( line = buf.readLine() ) != null ){
                pw.println( line );
            }
            p.waitFor();
            pw.close();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

    }
}

在调用此类的父线程中，我有：

while (commandList.size()>0) {
   String[] command = commandList.get(commandList.size()-1); // FILO queue
   Runnable runBash = new RunBashCommand( command, "folderAddress1", "folderAddress2" );
   Thread runBashThread = new Thread ( runBash );
   runBashThread.setName ( "some name" );
   runBashThread.start();
   RunCTPFS.threads.add(runBashThread); // my thread list for thread tracking
   commandList.remove(commandList.size()-1);
}

非常感激任何的帮助。我怀疑饿死了。

4

1 回答 1

2

  1. Does your bash script produce any output on stderr? It looks like you're reading its stdout via getInputStream() but you're not doing anything with getErrorStream(). If you don't read stderr then the process could hang if its stderr buffer fills up.

    Best practice when invoking processes is to read both stdout and stderr in separate threads. You must read them in parallel threads to avoid blocking.

  2. You don't need to have separate Java threads just for exec(). Each exec() call will start a separate process which executes in a separate thread of execution. The separate Java threads don't buy you anything. You can do all of the exec() calls from a single thread.

My recommendation: start all of the processes from a single thread. For each Process object you receive, start two background threads: one to process stdout, one to process stderr. Then in the original thread you can do waitFor() on each process in a loop.

List<Process> processes = new ArrayList<>();

for (String command: commands) {
    Process process = Runtime.getRuntime().exec(command);

    (new BackgroundReaderThread(process.getInputStream())).start();
    (new BackgroundReaderThread(process.getErrorStream())).start();

    processes.add(process);
}

for (Process process: processes) {
    process.waitFor();
}
于 2013-09-09T00:43:07.333 回答