Currently, I'm working on my first Convolutional Neural Network for a project in university. I have to create a model that can recognize if a cable has a defect by only using images of "good" and "defect" cables in Google Colab.
My dataset is unbalanced; I have more images of 'good' cables then 'defect' ones. That's why I used the imblearn-library and its function 'balanced_batch_generator' so that I could oversample the minority class. To work with the generator, I needed to reshape my X from dim 4 in dim 2. Now when I want to run the balanced batches in my model, I get an error due to the 2x2 shape and I don't know how to reshape the batches inside of the generator to get the model running.
I also tried to put a 'Flatten'-layer as the first layer with typed in input_shape to change the dense layer from the batch_generator but then I can't compile/build the model because it expects more dimensions than I can put in.
Here is my code:
#Model architecture
model = models.Sequential()
model.add(layers.SeparableConv2D(32,(3,3), activation='relu',
input_shape=(150,150,3)
)) #Number of filters = 32. Steigert sich mit jedem Con-layer mehr
#Standard Practice
model.add(layers.MaxPooling2D((2,2))) #MaxPooling: 2,2 => Reducing size of the images. Immer /2 und runter
#Standard Practice
model.add(layers.SeparableConv2D(64,(3,3), activation='relu',
input_shape=(150,150,3)
))
model.add(layers.MaxPooling2D((2,2)))
model.add(layers.SeparableConv2D(128,(3,3), activation='relu',
input_shape=(150,150,3)
))
model.add(layers.MaxPooling2D((2,2)))
model.add(layers.SeparableConv2D(128,(3,3), activation='relu',
input_shape=(150,150,3)
))
model.add(layers.MaxPooling2D((2,2)))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer=optimizers.RMSprop(lr=1e-4),
metrics=['acc'])
#lr = learning rate by 1.001
)
#Loading the images into the balanced_batch_generator
train_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size =(150,150),
batch_size = 9,
class_mode = 'binary'
from imblearn.keras import balanced_batch_generator
from imblearn.over_sampling import RandomOverSampler
for X, y in train_generator:
break
X = X.reshape(X.shape[0],-1)
training_generator, steps_per_epoch = balanced_batch_generator(X, y, sampler=RandomOverSampler(sampling_strategy='minority'), batch_size=8, keep_sparse = True, random_state=42)
callback_history = model.fit_generator(generator=training_generator, steps_per_epoch = steps_per_epoch,epochs=100, verbose=0)
Error message:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-159-c85c3a625f12> in <module>()
1 callback_history = model.fit_generator(generator=training_generator, steps_per_epoch = steps_per_epoch,
----> 2 epochs=100, verbose=0)
5 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
133 ': expected ' + names[i] + ' to have ' +
134 str(len(shape)) + ' dimensions, but got array '
--> 135 'with shape ' + str(data_shape))
136 if not check_batch_axis:
137 data_shape = data_shape[1:]
ValueError: Error when checking input: expected separable_conv2d_51_input to have 4 dimensions, but got array with shape (8, 67500)
Can anybody help me? Or give me a hint?