0

我试图在我的 linux 机器上使用 LPOT 量化一个训练有素的模型。通过以下链接

https://github.com/intel/lpot/tree/master/examples/helloworld/tf_example1

我在 conf.yaml 文件中指定了数据集路径，之后我尝试量化模型，但最终出现以下错误。

ValueError：在 --root 匹配中找不到文件：/home/u77217/.keras/datasets/fashion-mnist/ - -of- *

数据集文件夹包含以下文件：

  • t10k-images-idx3-ubyte.gz
  • t10k-labels-idx1-ubyte.gz
  • 火车图像-idx3-ubyte.gz
  • 火车标签-idx1-ubyte.gz

我的 conf.yaml 文件看起来像这样

#
# Copyright (c) 2021 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

version: 1.0

model:                                               # mandatory. used to specify model specific information.
  name: mobilenet_v1
  framework: tensorflow                              # mandatory. supported values are tensorflow, pytorch, pytorch_ipex, onnxrt_integer, onnxrt_qlinear or mxnet; allow new framework backend extension.

quantization:                                        # optional. tuning constraints on model-wise for advance user to reduce tuning space.
  calibration:
    sampling_size: 20                                # optional. default value is 100. used to set how many samples should be used in calibration.
    dataloader:
      dataset:
        ImageRecord:
          root: /home/u77217/.keras/datasets/fashion-mnist/            # NOTE: modify to calibration dataset location if needed
      transform:
        BilinearImagenet:
          height: 224
          width: 224

evaluation:                                          # optional. required if user doesn't provide eval_func in lpot.Quantization.
  accuracy:                                          # optional. required if user doesn't provide eval_func in lpot.Quantization.
    metric:
      topk: 1                                        # built-in metrics are topk, map, f1, allow user to register new metric.
    dataloader:
      batch_size: 32
      dataset:
        ImageRecord:
          root: /home/u77217/.keras/datasets/fashion-mnist/                   # NOTE: modify to evaluation dataset location if needed
      transform:
        BilinearImagenet:
          height: 224
          width: 224
4

1 回答 1

1

在数据集的 lpot YAML 文件中：如果您指定 ImageRecord 请以这种方式排列数据：

path/validation-000-of-100
path/validation-001-of-100
...
path/validation-099-of-100

文件名需要遵循这种模式：'* - * -of- *' 并且应该是 tf 记录形式。

在您的情况下，您打算使用时尚 mnist，您可以使用如下数据集配置

dataset:
  FashionMNIST:
  root: /path/fmnist
  train: False
  download: True

如果下载为True，它将下载数据集到/path/fmnist，否则用户应该把train-labels-idx1-ubyte.gz、train-images-idx3-ubyte.gz、t10k-labels-idx1-ubyte.gz和t10k -images-idx3-ubyte.gz 在 /path/fmnist 下手动。

在您的情况下，您可以使用root:/home/u77217/.keras/datasets/fashion-mnist/

请参考以下链接了解如何使用数据集进行 lpot 量化

https://github.com/intel/lpot/blob/cd8bdedadf185c0f0739dc7b232a415186b00578/docs/dataset.md

于 2021-09-13T05:58:26.667 回答