0

我正在使用迁移学习的概念对我的图像进行分类，我已经重用了https://towardsdatascience.com/transfer-learning-from-pre-trained-models-f2393f124751中提到的代码

该模型适用于我的 jupyter 笔记本中的数据（https://www.dropbox.com/s/esirpr6q1lsdsms/ricetransfer1.zip?dl=0），但在测试模型之前，我正在重塑图像。

但是当我想使用 TensorFlow 在浏览器中运行相同的模型时，我使用了 tfjs.converters.save_keras_model 来保存我的模型。

from keras.applications import VGG16
import tensorflowjs as tfjs
import tensorflow as tf
tf.compat.v1.disable_eager_execution()

img_width, img_height = 224, 224  # Default input size for VGG16

conv_base = VGG16(weights='imagenet', 
              include_top=False,
              #input_shape=(224, 224, 3))
              input_shape=(img_width, img_height, 3))

# Extract features
import os, shutil
from keras.preprocessing.image import ImageDataGenerator
import numpy as np
train_size, validation_size, test_size = 148, 27, 31

datagen = ImageDataGenerator(rescale=1./255)
batch_size = 16
train_dir = "ricetransfer1/train"
validation_dir = "ricetransfer1/validation"
test_dir="ricetransfer1/test"
#indices = np.random.choice(range(len(X_train)))

def extract_features(directory, sample_count):
   #sample_count= X_train.ravel()

    features = np.zeros(shape=(sample_count, 7, 7, 512))  # Must be 
    equal to the output of the convolutional base
    labels = np.zeros(shape=(sample_count))
    # Preprocess data
    generator = datagen.flow_from_directory(directory,
                                        target_size=(img_width,img_height),
                                        batch_size = batch_size,
                                        class_mode='binary')
    # Pass data through convolutional base
    i = 0
    for inputs_batch, labels_batch in generator:
        features_batch = conv_base.predict(inputs_batch)
        features[i * batch_size: (i + 1) * batch_size] = features_batch
        labels[i * batch_size: (i + 1) * batch_size] = labels_batch
        i += 1
        if i * batch_size >= sample_count:
            break
return features, labels

train_features, train_labels = extract_features(train_dir, train_size)  
validation_features, validation_labels = extract_features(validation_dir, validation_size)
test_features, test_labels = extract_features(test_dir, test_size)


from keras import models
from keras import layers
from keras import optimizers

epochs = 1

 ricemodel = models.Sequential()
 ricemodel.add(layers.Flatten(input_shape=(7,7,512)),)
 ricemodel.add(layers.Dense(256, activation='relu', input_dim=(7*7*512)))
 ricemodel.add(layers.Dropout(0.5))
 ricemodel.add(layers.Dense(1, activation='sigmoid'))
 ricemodel.summary()

 ricemodel.compile(optimizer=optimizers.Adam(),
          loss='binary_crossentropy',
          metrics=['acc'])

 import os
 history=ricemodel.fit(train_features, train_labels,
                epochs=epochs,
                batch_size=batch_size, 
                validation_data=(validation_features, validation_labels))

path='\vgg'
tfjs.converters.save_keras_model(ricemodel, path)

TensorFlowjs 代码是

 $(document).ready()
{
  $('.progress-bar').hide();
}
$("#image-selector").change(function(){
let reader = new FileReader();

reader.onload = function(){
    let dataURL = reader.result;
    $("#selected-image").attr("src",dataURL);
    $("#prediction-list").empty();
}
let file = $("#image-selector").prop('files')[0];
reader.readAsDataURL(file);
});


$("#model-selector").change(function(){
loadModel($("#model-selector").val());
$('.progress-bar').show();
})

let model;
async function loadModel(name){
model=await tf.loadModel(`http://localhost:8081/${name}/model.json`);
$('.progress-bar').hide();
}


$("#predict-button").click(async function(){
let image= $('#selected-image').get(0);
let tensor = preprocessImage(image,$("#model-selector").val());

let prediction = await model.predict(tensor).data();
let top5=Array.from(prediction)
            .map(function(p,i){
return {
    probability: p,
    className: IMAGENET_CLASSES[i]
};
}).sort(function(a,b){
    return b.probability-a.probability;
}).slice(0,5);

$("#prediction-list").empty();
top5.forEach(function(p){
$("#prediction- 
 list").append(`<li>${p.className}:${p.probability.toFixed(6)}</li>`);
});

});


function preprocessImage(image,modelName)
{
 let tensor=tf.fromPixels(image)
.resizeNearestNeighbor([224,224])
.toFloat();//.sub(meanImageNetRGB)

 if(modelName==undefined)
 {
    return tensor.expandDims();
}
else if(modelName=="vgg")
{
    let meanImageNetRGB= tf.tensor1d([123.68,116.779,103.939]);
    return tensor.sub(meanImageNetRGB)
                .reverse(2)
                .expandDims();
}
else if(modelName=="mobilenet")
{
    let offset=tf.scalar(127.5);
    return tensor.sub(offset)
                .div(offset)
                .expandDims();
}
else
{
    throw new Error("UnKnown Model error");
}
}

在 tensorflowjs 中加载模型后浏览器出现错误消息，我收到以下错误（我可以在 Web 开发控制台中看到错误消息）

未捕获（承诺中）错误：检查时出错：预期 flatten_1_input 的形状为 [null,7,7,512] 但得到的数组的形状为 [1,224,224,3]

有什么办法可以解决这个问题，我可以在加载到浏览器之前重塑图像吗？

我尝试了所有可能的选择，但我现在卡住了。

我已经在 stackoverflow 上检查了可能的解决方案。如何在浏览器上运行分类模型？

4

2 回答 2

1

正如@andyPotato的回答所指出的，您还需要将特征提取器模型转换为js。

from tensorflow.python.keras.applications.vgg16 import VGG16
from tensorflow.python.keras.models import save_model

#download model 
model = VGG16(weights='imagenet', 
          include_top=False,
          input_shape=(width, height, 3)) # tune parameters

#save the model 
save_model(
    model,
    "vgg.h5",
    overwrite=True,
)

将 VGG 特征提取器转换为 js

tensorflowjs_converter --input_format keras \
                       path/to/my_model.h5 \
                       path/to/tfjs_target_dir

现在在 js 中使用这两种模型进行推理

model1 = await tf.loadModel(`/url/of/vgg/converted/model.json`);

featureExtracted = await model1.predict(image)

model2 = await tf.loadModel(`/url/of/sequential/model/model.json`);

prediction = await model2.predict(featureExtracted)
于 2019-09-04T11:22:14.373 回答
1

ricemodel问题是您正在创建一个看起来像完全连接层的全新模型，并将其保存为独立模型，没有下面的卷积基础（在您的情况下为 VGG）。这就是为什么您的模型具有形状为 [,7,7,512]（特征向量）而不是 [,224,224,3]（原始图像数据）的输入层。

要解决这个问题，您需要首先加载具有预先训练的权重的 VGG 模型（例如“imagenet”），弹出顶层，然后将您ricemodel的添加到顶部。最后保存合并后的新模型并导出到 tfjs。

于 2019-09-04T01:07:29.900 回答