0

我正在尝试使用下面提到的代码通过 Python 脚本将文件写入 HDFS。

import hadoopy
import os

hdfs_path = 'data.json'


def read_local_dir(local_path):
    for fn in os.listdir(local_path):
        path = os.path.join(local_path, fn)
        if os.path.isfile(path):
            yield path, open(path).read()


def main():
    local_path = '.'
    hadoopy.writetb(hdfs_path, read_local_dir(local_path))

if __name__ == '__main__':
    main()

在这种情况下，为了执行上面的代码，我试图pip install hadoopy打包在cmd中，但安装失败。还尝试通过下载hadoopy-0.5.0.tar.gz进行安装，但没有成功。请帮我解决这个问题。

错误 ：

> pip install hadoopy
Collecting hadoopy
  Using cached hadoopy-0.5.0.tar.gz
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info\hadoopy.egg-info
    writing top-level names to pip-egg-info\hadoopy.egg-info\top_level.txt
    writing pip-egg-info\hadoopy.egg-info\PKG-INFO
    writing dependency_links to pip-egg-info\hadoopy.egg-info\dependency_links.t
xt
    writing manifest file 'pip-egg-info\hadoopy.egg-info\SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "<Path>\AppData\Local\Temp\pip-build-4mzcdtzr\hadoopy\setup.
py", line 95, in <module>
        ext_modules=ext_modules)
      File "c:\python34\lib\distutils\core.py", line 148, in setup
        dist.run_commands()
      File "c:\python34\lib\distutils\dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "c:\python34\lib\distutils\dist.py", line 974, in run_command
        cmd_obj.run()
      File "c:\python34\lib\site-packages\setuptools\command\egg_info.py", line
180, in run
        self.find_sources()
      File "c:\python34\lib\site-packages\setuptools\command\egg_info.py", line
207, in find_sources
        mm.run()
      File "c:\python34\lib\site-packages\setuptools\command\egg_info.py", line
291, in run
        self.add_defaults()
      File "c:\python34\lib\site-packages\setuptools\command\egg_info.py", line
320, in add_defaults
        sdist.add_defaults(self)
      File "c:\python34\lib\site-packages\setuptools\command\sdist.py", line 125
, in add_defaults
        for _, src_dir, _, filenames in build_py.data_files:
      File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line
59, in __getattr__
        self.data_files = files = self._get_data_files()
      File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line
86, in _get_data_files
        file[plen:] for file in self.find_data_files(package, src_dir)
      File "c:\python34\lib\site-packages\setuptools\command\build_py.py", line
94, in find_data_files
        + self.package_data.get(package, []))
    TypeError: can only concatenate list (not "map") to list

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in <Path>\A
ppData\Local\Temp\pip-build-4mzcdtzr\hadoopy\

提前致谢..！

4

0 回答 0