0

我正在尝试使用 TensorflowLite 在 Mali GPU 上部署 BERT 和 Transformer 模型。但问题是TensorflowLite不支持这些模型中的一些操作，包括{CAST, GATHER, MUL, RESHAPE, UNPACK}。有谁知道我如何在 GPU 上委派这些操作？是否有任何其他 TensorflowLite 库可以支持嵌入式 GPU，特别是 Mali GPU？我只想测量它们在 GPU 上的延迟。

STARTING!
Log parameter values verbosely: [0]
Min num runs: [1]
**Graph**: [mobilebert_float_384_gpu.tflite]
**Use gpu**: [1]
Loaded model mobilebert_float_384_gpu.tflite
**INFO**: Created TensorFlow Lite delegate for GPU.
**ERROR**: Following operations are not supported by GPU delegate:
CAST: Not supported cast case
GATHER: Operation is not supported.
MUL: MUL requires one tensor that not less than second in all dimensions.
RESHAPE: OP is supported, but tensor type isn't matched!
UNPACK: Operation is not supported.
**2661 operations will run on the GPU, and the remaining 81 operations will run on the CPU.**
INFO: Initialized OpenCL-based API.
INFO: Created 1 GPU delegate kernels.
Explicitly applied GPU delegate, and the model graph will be partially executed by the delegate w/ 1 delegate kernels.
The input model file size (MB): 100.239
Initialized session in 2491.9ms.
Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.
count=2 first=434022 curr=247839 min=247839 max=434022 avg=340930 std=93091
4

1 回答 1

0

GPU 委托不支持所有操作，仅支持此 list。

如果您可以使用NNAPI 委托，您可能希望尝试它。它可以将“最佳”委托给不同的处理器，而不仅仅是 GPU。支持的操作在这里。

最坏的情况是，如果这些都不适合您，要么回退到 CPU，要么编写自己的委托 - 但这是一大块工作。

于 2021-07-21T20:03:53.323 回答