我从远程 ftp 服务器下载文件(数千个文件)时遇到了很大的麻烦。下载几个文件没什么大不了的,但是当需要下载数千个文件时,我发现登录操作确实会减慢整个过程。我需要帮助以获得更好的脚本。
远程文件夹和文件结构类似于
CASE
--processor0
----0
------A.gz
------B.gz
------(and so on)
------H.gz
----0.1
------A.gz
------B.gz
------(and so on)
------H.gz
----0.2
------A.gz
------B.gz
------(and so on)
------H.gz
----(And so on with 0.3 0.4 0.5 ...)
--processor1
----0
------A.gz
------B.gz
------(and so on)
------H.gz
----0.1
------A.gz
------B.gz
------(and so on)
------H.gz
----0.2
------A.gz
------B.gz
------(and so on)
------H.gz
----(And so on with 0.3 0.4 0.5 ...)
.............(and so on)
--processor2000
----0
------A.gz
------B.gz
------(and so on)
------H.gz
----0.1
------A.gz
------B.gz
------(and so on)
------H.gz
----0.2
------A.gz
------B.gz
------(and so on)
------H.gz
----(And so on with 0.3 0.4 0.5 ...)
所以我写了一个shell脚本如下:
#!/bin/bash
stty -echo
read -p "Enter your password [ENTER]: " THISPASS; echo
stty echo
USERNAME="superman"
URL="supercomputer.org"
ROOTPATH="/home/remote/folder/CASE"
TIMELIST="0 0.1 0.2 0.3 0.4 0.5 0.6"
mkdir -p /home/LOCALFOLDER
LOCALDIR=/home/LOCALFOLDER
cd $LOCALDIR
echo ""
echo "Local Directory : "$LOCALDIR
echo "Remote URL : "$URL
echo "Remote Directory : "$ROOTPATH
echo ""
for TIMENAME in $TIMELIST; do
for (( i = 0; i < 1200; i++ )); do
CURPROC="processor"${i}
FILEPATH=$ROOTPATH/$CURPROC
mkdir -p $PWD/$CURPROC/$TIMENAME
cd $PWD/$CURPROC/$TIMENAME
echo -e "FOLDER - "$FILEPATH/$TIMENAME
FILEFULLNAMEA=$FILEPATH/$TIMENAME/"A.gz"
FILEFULLNAMEB=$FILEPATH/$TIMENAME/"B.gz"
FILEFULLNAMEC=$FILEPATH/$TIMENAME/"C.gz"
FILEFULLNAMED=$FILEPATH/$TIMENAME/"D.gz"
FILEFULLNAMEE=$FILEPATH/$TIMENAME/"E.gz"
FILEFULLNAMEF=$FILEPATH/$TIMENAME/"F.gz"
FILEFULLNAMEG=$FILEPATH/$TIMENAME/"G.gz"
FILEFULLNAMEH=$FILEPATH/$TIMENAME/"H.gz"
lftp sftp://$USERNAME:$THISPASS@$URL -e "get $FILEFULLNAMEA $FILEFULLNAMEB $FILEFULLNAMEC $FILEFULLNAMED $FILEFULLNAMEE $FILEFULLNAMEF $ FILEFULLNAMEG $FILEFULLNAMEH; bye"
cd ../..
done
done
echo ""
但我发现这lftp sftp://$USERNAME:$THISPASS@$URL -e "get $FILEFULLNAMEA $FILEFULLNAMEB $FILEFULLNAMEC $FILEFULLNAMED $FILEFULLNAMEE $FILEFULLNAMEF $ FILEFULLNAMEG $FILEFULLNAMEH; bye"
是下载过程中最耗时的部分。有没有更好的解决方案让我不需要一直登录?非常感谢!