0

我已经为自动 1) 启动 hadoop 服务 (namenode,datanode,jobtracker,tasktracker,secondary namenode) 编写了一个 shell 脚本，2) 从 hive 中删除所有表 3) 再次从 SQL SERVER 导入 hive 中的所有表

我从java调用这个shell脚本。下面是Shell Script和Java Code的代码

外壳脚本：

export HADOOP_HOME=/home/hadoop/hadoop-0.20.2-cdh3u2/
export HIVE_HOME=/home/hadoop/hive-0.7.1/
export SQOOP_HOME=/home/hadoop/sqoop-1.3.0-cdh3u1/
export MSSQL_CONNECTOR_HOME=/home/hadoop/sqoop-sqlserver-1.0
export HBASE_HOME=/home/hadoop/hbase-0.90.1-cdh3u0
export ZOOKEEPER_HOME=/home/hadoop/zookeeper-3.3.1+10
export SQOOP_CONF_DIR=/home/hadoop/sqoop-1.3.0-cdh3u1/conf/

/home/hadoop/hadoop-0.20.2-cdh3u2/bin/hadoop/start-all.sh
/home/hadoop/hadoop-0.20.2-cdh3u2/bin/hadoop -rmr /user/hadoop/*

/home/hadoop/hive-0.7.1/bin/hive -e 'show tables' > TablesToDelete.txt
while read line1
do
    echo 'drop table '$line1
    /home/hadoop/hive-0.7.1/bin/hive -e 'drop table '$line1
done < TablesToDelete.txt

while read line
do
    echo $line" ------------------------------"
/home/hadoop/sqoop-1.3.0-cdh3u1/bin/sqoop-import --connect 'jdbc:sqlserver://192.168.1.1;username=abcd;password=12345;database=HadoopTest' --table line --hive-table $line  --create-hive-table --hive-import -m 1 --hive-drop-import-delims --hive-home /home/hadoop/hive-0.7.1 --verbose
done < /home/hadoop/sqoop-1.3.0-cdh3u1/bin/tables.txt

Java 代码：

public class ImportTables
{

    public static void main(String arsg[])
    {
        PrintWriter pw=null;
        try
        {
            Formatter formatter = new Formatter();
            String LogFile = "Log-"+ formatter.format("%1$tm%1$td-%1$tH%1$tM%1$tS", new Date());   
            File f=new File("/home/hadoop/"+LogFile);
            FileWriter fw1=null;   
            pw=new PrintWriter(f);

            String cmd = "/home/hadoop/sqoop-1.3.0-cdh3u1/bin/TablesToImport.sh"; // this is the command to execute in the Unix shell

            // create a process for the shell
            ProcessBuilder pb = new ProcessBuilder("bash", "-c", cmd);
            pb.redirectErrorStream(true); // use this to capture messages sent to stderr
            Process shell = pb.start();
            InputStream shellIn = shell.getInputStream(); // this captures the output from the command
            int shellExitStatus = shell.waitFor();
            // wait for the shell to finish and get the return code
            // at this point you can process the output issued by the command

            // for instance, this reads the output and writes it to System.out:
            int c;
            while ((c = shellIn.read()) != -1)
            {
                System.out.write(c);
            }

            // close the stream
            shellIn.close();
        }
        catch(Exception e)
        {
            e.printStackTrace();
            e.printStackTrace(pw);
            pw.flush();
            System.exit(1);
        }



    }
}

但是当我运行程序时，我在控制台上什么也看不到，程序仍处于运行模式。如果我把下面的代码离子外壳脚本：

/home/hadoop/hive-0.7.1/bin/hive -e 'show tables' > TablesToDelete.txt
while read line1
do
    echo 'drop table '$line1
    /home/hadoop/hive-0.7.1/bin/hive -e 'drop table '$line1
done < TablesToDelete.txt

然后输出如下：

找不到 hadoop 安装：必须设置 $HADOOP_HOME 或 hadoop 必须在路径中

我的程序/脚本有什么问题？在哪里以及如何在我的脚本中设置 HADOOP_HOME 和所有路径？

4

1 回答 1

1

waitFor顾名思义，调用是阻塞调用。它会停止进一步的执行，直到该过程完成。但是由于您的代码也是进程标准输出的接收器，因此整个过程都会阻塞。处理完脚本的输出后，只需将 移至waitFor。

于 2012-04-18T09:01:56.753 回答