0

Below is my shell script from which I am running two Hive SQL queries. Which is working fine.

#!/bin/bash

DATE_YEST_FORMAT2=`perl -e 'use POSIX qw(strftime); print strftime "%Y%m%d",localtime(time()- 3600*96);'`
echo $DATE_YEST_FORMAT2

hive -e "
set mapred.job.queue.name=data-technology;
SELECT SUM(total_items_purchased), SUM(total_items_missingormismatch) from lip_data_quality where dt='$DATE_YEST_FORMAT2';
SELECT 100 * SUM(total_items_missingormismatch*1.0) / SUM(total_items_purchased) FROM lip_data_quality where dt='$DATE_YEST_FORMAT2';"

I am running the above shell script like this-

sh -x test.sh

Problem Statement:-

If you see my first SELECT query, it will give me two columns as an OUTPUT, SUM of total_items_purchased and SUM of total_items_missingormismatch. And my second SELECT will give the percentage of those columns.

I need to send these three details in an email using that above SHELL SCRIPT. Basically I need to send these three things in an email.

From the first query- I need to send-

total_items_purchased and total_items_missingormismatch

And from the second query- I need to send-

Percentage that I am calculating.

Email can be like this or some better approach-

Total Items Purchased:- Some Number
Total Items MissingorMismatch:- Some Number
Error Percentage:- Some Percentage

Any suggestions will be appreciated.

Update:

After making changes in my Shell Script, I ran the below shell script-

#!/bin/bash

QUERY1 = `hive -e "
set mapred.job.queue.name=data-technology;
SELECT SUM(total_items_purchased), SUM(total_items_missingormismatch) from lip_data_quality where dt='20120804';`

QUERY2 = `hive -e "
set mapred.job.queue.name=data-technology;
SELECT 100 * SUM(total_items_missingormismatch*1.0) / SUM(total_items_purchased) FROM lip_data_quality where dt='20120804';"`

echo "Total items purchased: `echo $QUERY1 | awk '{print $1}'`"
echo "Total Items MissingorMismatch: `echo $QUERY1 | awk '{print $2}'`"
echo "Error Percentage: $QUERY2"

Output that I got which I don't think is right? Something wrong with script? As I am running SunOS:

bash-3.00$ sh -x emailtest.sh
+ hive -e
set mapred.job.queue.name=data-technology;
SELECT SUM(total_items_purchased), SUM(total_items_missingormismatch) from lip_data_quality where dt='20120804';
+ QUERY1 = 3443837 448750
emailtest.sh: QUERY1: not found
+ hive -e
set mapred.job.queue.name=data-technology;
SELECT 100 * SUM(total_items_missingormismatch*1.0) / SUM(total_items_purchased) FROM lip_data_quality where dt='20120804';
+ QUERY2 = 13.030523802375084
emailtest.sh: QUERY2: not found
+ awk {print $1}
+ echo
+ echo Total items purchased:
Total items purchased:
+ awk {print $2}
+ echo
+ echo Total Items MissingorMismatch:
Total Items MissingorMismatch:
+ echo Error Percentage:
Error Percentage:

Few parts from the script are working fine, but the last three echo statements are not working fine I guess.

4

1 回答 1

1

My solution, which is admittedly old-school, would be to use awk to parse the output by appending the following to your script:

QUERY1=`hive -e "
set mapred.job.queue.name=hdmi-technology;
SELECT SUM(total_items_purchased), SUM(total_items_missingormismatch) from lip_data_quality where dt='$DATE_YEST_FORMAT2';`

QUERY2=`hive -e "
set mapred.job.queue.name=hdmi-technology;
SELECT 100 * SUM(total_items_missingormismatch*1.0) / SUM(total_items_purchased) FROM lip_data_quality where dt='$DATE_YEST_FORMAT2';"`

echo "Total items purchased: `echo $QUERY1 | awk '{print $1}'`"
echo "Total Items MissingorMismatch: `echo $QUERY1 | awk '{print $2}'`"
echo "Error Percentage: $QUERY2"

These days, I'd do something like this in Python. But Awk still has its uses.

于 2012-08-08T22:00:51.963 回答