0

我的代码如下：

df['O_ID'] = (df.apply(get_match_id, arguments=[df['pickup_longitude'], df['pickup_latitude']])).jit_cuda()

第一次使用这个函数时——<strong>jit_cuda()，报错“No Module named cupy”

但是，当我安装了 cupy-cuda101（适应我的 CUDA 版本）时，我收到了一个新错误

Traceback (most recent call last):
  File "F:\Anaconda3\lib\site-packages\vaex\dataframe.py", line 3580, in table_part
    values[name] = df.evaluate(name)
  File "F:\Anaconda3\lib\site-packages\vaex\dataframe.py", line 2616, in evaluate
    return self._evaluate_implementation(expression, i1=i1, i2=i2, out=out, selection=selection, filtered=filtered, internal=internal, parallel=parallel, chunk_size=chunk_size)
  File "F:\Anaconda3\lib\site-packages\vaex\dataframe.py", line 5352, in _evaluate_implementation
    dtypes[expression] = df.data_type(expression, internal=False)
  File "F:\Anaconda3\lib\site-packages\vaex\dataframe.py", line 1998, in data_type
    data = self.evaluate(expression, 0, 1, filtered=True, internal=True, parallel=False)
  File "F:\Anaconda3\lib\site-packages\vaex\dataframe.py", line 2616, in evaluate
    return self._evaluate_implementation(expression, i1=i1, i2=i2, out=out, selection=selection, filtered=filtered, internal=internal, parallel=parallel, chunk_size=chunk_size)
  File "F:\Anaconda3\lib\site-packages\vaex\dataframe.py", line 5427, in _evaluate_implementation
    value = scope.evaluate(expression)
  File "F:\Anaconda3\lib\site-packages\vaex\scopes.py", line 97, in evaluate
    result = self[expression]
  File "F:\Anaconda3\lib\site-packages\vaex\scopes.py", line 139, in __getitem__
    self.values[variable] = self.evaluate(expression)  # , out=self.buffers[variable])
  File "F:\Anaconda3\lib\site-packages\vaex\scopes.py", line 103, in evaluate
    result = eval(expression, expression_namespace, self)
  File "<string>", line 1, in <module>
  File "F:\Anaconda3\lib\site-packages\vaex\expression.py", line 1073, in __call__
    return self.f(*args, **kwargs)
  File "F:\Anaconda3\lib\site-packages\vaex\expression.py", line 1120, in wrapper
    return cupy.asnumpy(func(*args))
  File "cupy\core\fusion.pyx", line 905, in cupy.core.fusion.Fusion.__call__
  File "cupy\core\fusion.pyx", line 754, in cupy.core.fusion._FusionHistory.get_fusion
  File "<string>", line 6, in f
NameError: name 'lambda_function_1' is not defined

我该如何解决？

4

1 回答 1

0

我的理解是，vaex 中的即时编译仅适用于虚拟列，或者主要通过使用 numpy 方法或纯 python 算术的各种算术运算计算的表达式/列。

使用的时候apply，一个函数可以很抽象，基本上是你想要的任何东西，所以它可能无法编译。

如果您可以.apply使用 numpy 表达式重写您的函数，那么您很可能可以使用该jit_cuda方法来加速它。Vaex 无论如何都不推荐使用.apply，因为它很难并行化，应该作为“最后的手段”使用。

来源：https ://vaex.io/docs/tutorial.html#Just-In-Time-compilation

于 2020-11-04T23:51:03.243 回答