我收到多模式问题的错误。输入形状：img 输入：- (3740, 150, 150, 3)，字输入：- (3740, 260) 其中 3740 是样本数。这里已附加模型作为函数，其中 build_img_encoder 描述 IMG 编码器模型， build_wrd_conv 描述 word_encoder 部分，它们是模型图图像中可见的 2 个输入分支
def build_merged_model_Concatenate(embedding_matrix, input_length):#input_length-->word length
#call encoders
#define input 2 channels
img_input = Input(shape= build_img_encoder_model().input_shape[1:], name="img_input")
wrd_input = Input(shape= build_wrd_conv(embedding_matrix ,input_length).input_shape[1:], name="wrd_input")
#call img and word encoders
img_encoder = build_img_encoder_model()(img_input)
wrd_encoder = build_wrd_conv(embedding_matrix, input_length)(wrd_input)
#merge 2 models
merge = concatenate([img_encoder, wrd_encoder], name="concat_merge1")
merged_model1_concat = keras.models.Model(inputs=[img_input, wrd_input], outputs = merge)## model assign
dense_merge0 = Dense(1024, activation='relu',name="dense_merge0")(merge)
dropout_merge0 = Dropout(0.5, name='dropout_merge0')(dense_merge0)
dense_merge1 = Dense(256, activation='relu', name='dense_merge1')(dropout_merge0)
dropout_3 = Dropout(0.2,name="dropout_3")(dense_merge1)
dense_merge2 = Dense(128, activation='relu', name='dense_merge_2')(dropout_3)
output_classify = Dense( 2, activation='sigmoid')(dense_merge2)
merged_classify = keras.models.Model(inputs=[img_input, wrd_input], outputs= output_classify)## model assign
OPTIMIZER = Adam(learning_rate=0.0001,beta_1=0.9, beta_2=0.999)
METRICS = ['accuracy',f1_m, precision_m, recall_m]
merged_model1_concat.compile(loss = 'binary_crossentropy', optimizer=OPTIMIZER, metrics=METRICS)
merged_classify.compile(loss = 'binary_crossentropy', optimizer=OPTIMIZER, metrics=METRICS)
return merged_model1_concat, merged_classify
在运行模型拟合模型开始训练并运行几乎一个完整的时期，然后出现此错误：-
WARNING:tensorflow:`input_shape` is undefined or non-square, or `rows` is not in [128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
WARNING:tensorflow:`input_shape` is undefined or non-square, or `rows` is not in [128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
Epoch 1/100
2/53 [>.............................] - ETA: 6s - loss: 1.0590 - accuracy: 0.6094 - f1_m: 0.5766 - precision_m: 0.6011 - recall_m: 0.5547WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time (batch time: 0.0632s vs `on_train_batch_end` time: 0.1851s). Check your callbacks.
53/53 [==============================] - ETA: 0s - loss: 0.8733 - accuracy: 0.6548 - f1_m: 0.6309 - precision_m: 0.6285 - recall_m: 0.6362WARNING:tensorflow:Model was constructed with shape (None, 150, 150, 3) for input Tensor("img_input_15:0", shape=(None, 150, 150, 3), dtype=float32), but it was called on an input with incompatible shape (None, 260).
WARNING:tensorflow:Model was constructed with shape (None, 150, 150, 3) for input Tensor("input_54:0", shape=(None, 150, 150, 3), dtype=float32), but it was called on an input with incompatible shape (None, 260).
WARNING:tensorflow:Model was constructed with shape (None, 150, 150, 3) for input Tensor("input_53:0", shape=(None, 150, 150, 3), dtype=float32), but it was called on an input with incompatible shape (None, 260).
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-84-23200389ca70> in <module>()
26 x, concat_merge_classify = build_merged_model_Concatenate(embedding_matrix_full, input_length)
27 ##plot_model(Classify_Model1, to_file="MergedClassify_Model_2_multiply.png",show_shapes=True)
---> 28 hist2= concat_merge_classify.fit(x=[X_TRAIN_, padded_docs_train] , y=Y_train, epochs=100, batch_size =64, callbacks=[reducelr, checkpointer,earlyStopping], validation_data = ([padded_docs_test],Y_test))
29 print('\n# Evaluate on test data')
30 loss, accuracy, f1_score, precision, recall= concat_merge_classify.evaluate([X_TEST_, padded_docs_test], Y_test, verbose=1, batch_size=64)
12 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
971 except Exception as e: # pylint:disable=broad-except
972 if hasattr(e, "ag_error_metadata"):
--> 973 raise e.ag_error_metadata.to_exception(e)
974 else:
975 raise
ValueError: in user code:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1224 test_function *
return step_function(self, iterator)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1215 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica
return fn(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1208 run_step **
outputs = model.test_step(data)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:1174 test_step
y_pred = self(x, training=False)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:985 __call__
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:386 call
inputs, training=training, mask=mask)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:508 _run_internal_graph
outputs = node.layer(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:985 __call__
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:386 call
inputs, training=training, mask=mask)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:508 _run_internal_graph
outputs = node.layer(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:985 __call__
outputs = call_fn(inputs, *args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:386 call
inputs, training=training, mask=mask)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/functional.py:508 _run_internal_graph
outputs = node.layer(*args, **kwargs)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:976 __call__
self.name)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py:180 assert_input_compatibility
str(x.shape.as_list()))
ValueError: Input 0 of layer conv1_pad is incompatible with the layer: expected ndim=4, found ndim=2. Full shape received: [None, 260]
那么发生了什么事？，看起来我在某个地方错过了一些形状。