我们重新启动了运行气流服务的服务器。一切正常，但由于无法访问 Web 服务器 UI，气流 Gunicorn 服务未运行。
网络服务器日志：每当我们启动网络服务器时，它就会卡住：
airflow webserver
[2021-12-22 18:26:15,403] {init.py:57} INFO - Using executor LocalExecutor
[2021-12-22 18:26:15,477] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2021-12-22 18:26:15,509] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
____ |( )_______ / /________ __
____ /| |_ /__ / / __ / __ _ | /| / /
___ ___ | / _ / _ / _ / / // / |/ |/ /
// |// // // // _/____/|__/
/usr/local/lib/python2.7/dist-packages/flask/exthook.py:71: ExtDeprecationWarning:
Importing flask.ext.cache is deprecated, use flask_cache instead.
.format(x=modname), ExtDeprecationWarning
[2021-12-22 18:26:15,906] [9263] {models.py:167} INFO - Filling up the DagBag from
/usr/local/lib/airflow/dags
/usr/local/lib/python2.7/dist-packages/airflow/utils/helpers.py:406:
DeprecationWarning: Importing SSHHook directly from <module 'airflow.contrib.hooks'
from '/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/init.pyc'> has
been deprecated. Please import from '<module 'airflow.contrib.hooks' from
'/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/init.pyc'>.
[operator_module]' instead. Support for direct imports will be dropped entirely in
Airflow 2.0.
DeprecationWarning)
[2021-12-22 18:26:17,439] [9263] {models.py:1958} WARNING - schedule_interval is used
for <Task(BashOperator): sync_dag>, though it has been deprecated as a task
parameter, you need to specify it as a DAG parameter instead
spark-submit --files /etc/spark2/conf/hive-site.xml --deploy-mode cluster --name
trust_ride_ano_in_1 --executor-memory 4g --queue trust --num-executors 5 --master
yarn --driver-memory 10G --py-files /home/suvratjain/RideAnomalyDetection.zip --conf
spark.ui.showConsoleProgress=true --conf spark.driver.maxResultSize=6g --conf
spark.files.overwrite=true /home/suvratjain/RideAnomalyDetection/raw_data.py --
country IN --s3_path s3a://s3-emr-test-
stg/fraud/ride_level_anomaly/raw_feature/IN/{{
task_instance.xcom_pull(key="for_date", task_ids='set_for_date') }} --day_t {{
task_instance.xcom_pull(key="for_date", task_ids='set_for_date') }}
/usr/local/lib/python2.7/dist-packages/airflow/models.py:1927:
PendingDeprecationWarning: Invalid arguments were passed to BashOperator. Support
for passing such arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {'provide_context': True}
category=PendingDeprecationWarning
spark-submit --files /etc/spark2/conf/hive-site.xml --deploy-mode cluster --name
trust_ride_ano_gb_1 --executor-memory 4g --queue trust --num-executors 5 --master
yarn --driver-memory 10G --py-files /home/suvratjain/RideAnomalyDetection.zip --
conf spark.ui.showConsoleProgress=true --conf spark.driver.maxResultSize=6g --conf
spark .files.overwrite=true /home/suvratjain/RideAnomalyDetection/raw_data.py --
country GB --s3_path s3a://s3-emr-test-
stg/fraud/ride_level_anomaly/raw_feature/GB/{{
task_instance.xcom_pull(key="for_date", task_ids='set_for_date') }} --day_t {{
task_instance.xcom_pull(key="for_date", task_ids='set_for_date') }}
PendingDeprecationWarning:
Invalid arguments were passed to SparkSubmitOperator. Support for passing such
arguments will be dropped in Airflow 2.0. Invalid arguments were:
*args: ()
**kwargs: {'driver_memory': '2g', 'application_args': ['staging', '2021-12-22'],
'java_class': 'com.ola.fs.preauth.exps.Exps1'} category=PendingDeprecationWarning
此行后没有任何反应。
现在，当尝试启动 gunicorn 时出现 App not found 错误：
root@airflow1:/usr/local/lib/python2.7/dist-packages/airflow# gunicorn -b
0.0.0.0:8080 airflow
[2021-12-22 21:11:03 +0000] [21851] [INFO] Starting gunicorn 19.3.0
[2021-12-22 21:11:03 +0000] [21851] [INFO] Listening at: http://0.0.0.0:8080 (21851)
[2021-12-22 21:11:03 +0000] [21851] [INFO] Using worker: sync
[2021-12-22 21:11:03 +0000] [21856] [INFO] Booting worker with pid: 21856
[2021-12-22 21:11:03,753] {init.py:57} INFO - Using executor LocalExecutor
[2021-12-22 21:11:03,840] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2021-12-22 21:11:03,860] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Failed to find application: 'airflow'
[2021-12-22 21:11:03 +0000] [21856] [INFO] Worker exiting (pid: 21856)
[2021-12-22 21:11:03 +0000] [21851] [INFO] Shutting down: Master
[2021-12-22 21:11:03 +0000] [21851] [INFO] Reason: App failed to load.
root@airflow1:/usr/local/lib/python2.7/dist-packages/airflow#
- 气流版本：1.8.0
- 独角兽版本：19.3.0
- Python版本：2.7