1
4

2 回答 2

0

有一个名为的二进制文件make_image_db.cc正是您所描述的。它位于caffe2/build/bin/make_image_db：

// This script converts an image dataset to a database.
//
// caffe2::FLAGS_input_folder is the root folder that holds all the images
//
// caffe2::FLAGS_list_file is the path to a file containing a list of files
// and their labels, as follows:
//
//   subfolder1/file1.JPEG 7
//   subfolder1/file2.JPEG 7
//   subfolder2/file1.JPEG 8
//   ...

如https://github.com/caffe2/caffe2/issues/1755中所述，您可以通过以下方式使用二进制文件（也可以使用更少的参数）：

caffe2/build/bin/make_image_db -color -db lmdb -input_folder ./some_input_folder
-list_file ./labels_file -num_threads 10 -output_db_name ./some_output_folder -raw -scale 256 -shuffle

关于如何创建和读取 lmdb 数据库（用于随机图像）的完整 Caffe2 示例可以在官方 github 存储库中找到，并且可以用作框架以适应您自己的图像https://github.com/caffe2/caffe2 /blob/master/caffe2/python/examples/lmdb_create_example.py。由于我还没有使用过这种方法，所以我将简单地复制示例。为了创建数据库，可以使用：

import argparse
import numpy as np

import lmdb
from caffe2.proto import caffe2_pb2
from caffe2.python import workspace, model_helper

def create_db(output_file):
    print(">>> Write database...")
    LMDB_MAP_SIZE = 1 << 40   # MODIFY
    env = lmdb.open(output_file, map_size=LMDB_MAP_SIZE)

    checksum = 0
    with env.begin(write=True) as txn:
        for j in range(0, 128):
            # MODIFY: add your own data reader / creator
            label = j % 10
            width = 64
            height = 32

            img_data = np.random.rand(3, width, height)
            # ...

            # Create TensorProtos
            tensor_protos = caffe2_pb2.TensorProtos()
            img_tensor = tensor_protos.protos.add()
            img_tensor.dims.extend(img_data.shape)
            img_tensor.data_type = 1

            flatten_img = img_data.reshape(np.prod(img_data.shape))
            img_tensor.float_data.extend(flatten_img)

            label_tensor = tensor_protos.protos.add()
            label_tensor.data_type = 2
            label_tensor.int32_data.append(label)
            txn.put(
                '{}'.format(j).encode('ascii'),
                tensor_protos.SerializeToString()
            )

            checksum += np.sum(img_data) * label
            if (j % 16 == 0):
                print("Inserted {} rows".format(j))

    print("Checksum/write: {}".format(int(checksum)))
    return checksum

然后可以通过以下方式加载数据库：

def read_db_with_caffe2(db_file, expected_checksum):
    print(">>> Read database...")
    model = model_helper.ModelHelper(name="lmdbtest")
    batch_size = 32
    data, label = model.TensorProtosDBInput(
        [], ["data", "label"], batch_size=batch_size,
        db=db_file, db_type="lmdb")

    checksum = 0

    workspace.RunNetOnce(model.param_init_net)
    workspace.CreateNet(model.net)

    for _ in range(0, 4):
        workspace.RunNet(model.net.Proto().name)

        img_datas = workspace.FetchBlob("data")
        labels = workspace.FetchBlob("label")
        for j in range(batch_size):
            checksum += np.sum(img_datas[j, :]) * labels[j]

    print("Checksum/read: {}".format(int(checksum)))
    assert np.abs(expected_checksum - checksum < 0.1), \
        "Read/write checksums dont match"

最后但同样重要的是，还有一个关于如何创建 minidb 数据库的教程：https ://github.com/caffe2/caffe2/blob/master/caffe2/python/tutorials/create_your_own_dataset.ipynb 。为此，可以使用以下功能：

def write_db(db_type, db_name, features, labels):
    db = core.C.create_db(db_type, db_name, core.C.Mode.write)
    transaction = db.new_transaction()
    for i in range(features.shape[0]):
        feature_and_label = caffe2_pb2.TensorProtos()
        feature_and_label.protos.extend([
            utils.NumpyArrayToCaffe2Tensor(features[i]),
            utils.NumpyArrayToCaffe2Tensor(labels[i])])
        transaction.put(
            'train_%03d'.format(i),
            feature_and_label.SerializeToString())
    # Close the transaction, and then close the db.
    del transaction
    del db

特征将是一个张量，其中包含您的图像作为 numpy 数组。标签是特征对应的真实标签。然后，您只需将该函数称为

write_db("minidb", "train_images.minidb", train_features, train_labels)

最后，您将通过以下方式从数据库中加载图像

net_proto = core.Net("example_reader")
dbreader = net_proto.CreateDB([], "dbreader", db="train_images.minidb", db_type="minidb")
net_proto.TensorProtosDBInput([dbreader], ["X", "Y"], batch_size=16)
于 2018-02-13T18:49:19.713 回答
-1

在 lmbd 中创建数据库： 创建火车数据文件夹 创建包含文件名标签的 train.txt 文件 创建验证数据文件夹 创建包含文件名和标签的 val.txt 文件

编辑这个文件 

gedit examples/imagenet/create_imagenet.sh


EXAMPLE= path to where *.lmbd folder wil be stored
DATA= path where val.txt and train.txt is present
TOOLS=build/tools

TRAIN_DATA_ROOT=test/make_caffe_data/train/ # path to trainfiles
VAL_DATA_ROOT=test/make_caffe_data/val/ # path to test_files

设置RESIZE=true为将图像大小调整为 256x256。就false好像图像已经使用其他工具调整大小一样。 RESIZE=true

./examples/imagenet/create_imagenet.sh

于 2018-05-17T01:49:14.407 回答