2

我是 DL 和 Keras 的新手，目前我正在尝试在 Keras 中实现一个基于 sobel-filter 的自定义损失函数。

这个想法是计算索贝尔滤波预测和索贝尔滤波地面实况图像的均方损失。

到目前为止，我的自定义损失函数如下所示：

from scipy import ndimage

def mse_sobel(y_true, y_pred):

    for i in range (0, y_true.shape[0]):
        dx_true = ndimage.sobel(y_true[i,:,:,:], 1)
        dy_true = ndimage.sobel(y_true[i,:,:,:], 2)
        mag_true[i,:,:,:] = np.hypot(dx_true, dy_true)
        mag_true[i,:,:,:] *= 1.0 / np.max(mag_true[i,:,:,:])

        dx_pred = ndimage.sobel(y_pred[i,:,:,:], 1)
        dy_pred = ndimage.sobel(y_pred[i,:,:,:], 2)
        mag_pred[i,:,:,:] = np.hypot(dx_pred, dy_pred)
        mag_pred[i,:,:,:] *= 1.0 / np.max(mag_pred[i,:,:,:])

    return(K.mean(K.square(mag_pred - mag_true), axis=-1))

使用这个损失函数会导致这个错误：

in mse_sobel
for i in range (0, y_true.shape[0]):
TypeError: __index__ returned non-int (type NoneType)

使用我发现的调试器，它y_true.shape只会返回None- 很好。但是当我替换y_true.shape为例如1它看起来像这样for i in range (0,1):时，会发生另一个错误：

in sobel
axis = _ni_support._check_axis(axis, input.ndim)

in _check_axis
raise ValueError('invalid axis')
ValueError: invalid axis

在这里，我不确定为什么轴似乎无效？

谁能帮我弄清楚如何实现该损失函数？非常感谢您的帮助！

4

1 回答 1

8

Losses must be made with tensor operations, using the keras backend, or tensorflow/theano/cntk functions. This is the only way to keep backpropagation. Using numpy, scipy etc. breaks the graph.

Let's import the keras backend:

import keras.backend as K

Defining the filters:

#this contains both X and Y sobel filters in the format (3,3,1,2)
#size is 3 x 3, it considers 1 input channel and has two output channels: X and Y
sobelFilter = K.variable([[[[1.,  1.]], [[0.,  2.]],[[-1.,  1.]]],
                      [[[2.,  0.]], [[0.,  0.]],[[-2.,  0.]]],
                      [[[1., -1.]], [[0., -2.]],[[-1., -1.]]]])

Here, a function that repeats the filters for each input channel, in case your images are RGB or have more than 1 channel. This will just replicate the sobel filters for each input channel: (3,3,inputChannels, 2):

def expandedSobel(inputTensor):

    #this considers data_format = 'channels_last'
    inputChannels = K.reshape(K.ones_like(inputTensor[0,0,0,:]),(1,1,-1,1))
    #if you're using 'channels_first', use inputTensor[0,:,0,0] above

    return sobelFilter * inputChannels

And this is the loss function:

def sobelLoss(yTrue,yPred):

    #get the sobel filter repeated for each input channel
    filt = expandedSobel(yTrue)

    #calculate the sobel filters for yTrue and yPred
    #this generates twice the number of input channels 
    #a X and Y channel for each input channel
    sobelTrue = K.depthwise_conv2d(yTrue,filt)
    sobelPred = K.depthwise_conv2d(yPred,filt)

    #now you just apply the mse:
    return K.mean(K.square(sobelTrue - sobelPred))

Apply this loss in the model:

model.compile(loss=sobelLoss, optimizer = ....)

My experience shows that calculating the unified sobel filter sqrt(X² + Y²) brings terrible results and the resulting images sound like chess boards. But if you do want it:

def squareSobelLoss(yTrue,yPred):

    #same beginning as the other loss
    filt = expandedSobel(yTrue)
    squareSobelTrue = K.square(K.depthwise_conv2d(yTrue,filt))
    squareSobelPred = K.square(K.depthwise_conv2d(yPred,filt))

    #here, since we've got 6 output channels (for an RGB image)
    #let's reorganize in order to easily sum X² and Y²: change (h,w,6) to (h,w,3,2)
    #caution: this method of reshaping only works in tensorflow
    #if you do need this in other backends, let me know
    newShape = K.shape(squareSobelTrue)
    newShape = K.concatenate([newShape[:-1],
                              newShape[-1:]//2,
                              K.variable([2],dtype='int32')])

    #sum the last axis (the one that is 2 above, representing X² and Y²)                      
    squareSobelTrue = K.sum(K.reshape(squareSobelTrue,newShape),axis=-1)
    squareSobelPred = K.sum(K.reshape(squareSobelPred,newShape),axis=-1)

    #since both previous values are already squared, maybe we shouldn't square them again? 
    #but you can apply the K.sqrt() in both, and then make the difference, 
    #and then another square, it's up to you...    
    return K.mean(K.abs(squareSobelTrue - squareSobelPred))
于 2017-11-17T11:55:07.573 回答