0

我在 Tensorflow 中实现 RBF 网络时遇到问题。我需要计算 x 和质心之间的欧几里得距离（来自 RBF newtork 的定义）。我写了这段代码：

    x_data = tf.placeholder(shape=[None, 3], dtype=tf.float32)
    y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)

    # Create variables for NN layers
    A1 = tf.Variable(tf.random_normal(shape=[3, first_layer_nodes]))  # input -> first layer nodes
    A2 = tf.Variable(tf.random_normal(shape=[first_layer_nodes, 1]))  # first_layer nodes -> sum node
    c = tf.Variable(tf.random_normal(shape=[first_layer_nodes]))  # centroids

    # Declare NN
    inputs_with_weights = tf.matmul(x_data, A1)
    print(inputs_with_weights)
    # euclid_dist = tf.sqrt(tf.reduce_sum(tf.pow(tf.subtract(inputs_with_weights, c), 2)))
    euclid_dist = tf.norm(inputs_with_weights - c, ord='euclidean')
    print(euclid_dist)
    first_output = tf_gaussian_function(euclid_dist)
    print(first_output)

final_output = tf.matmul(first_output, A2)

但我是这样的问题：

E:\#PROJEKTY\#PROGRAMOWANIE\AI-Project>python Iris.py
2018-04-27 00:49:37.800684: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
<tf.Variable 'Variable_2:0' shape=(1, 1) dtype=float32_ref>
Tensor("MatMul:0", shape=(?, 1), dtype=float32)
Tensor("norm/Squeeze:0", shape=(), dtype=float32)
Tensor("gaussian_function:0", dtype=float32)
Traceback (most recent call last):
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1361, in _do_call
    return fn(*args)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _run_fn
    target_list, status, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Iris.py", line 144, in <module>
    sess.run(train_step, feed_dict={x_data: x_d, y_target: y_d})
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 905, in run
    run_metadata_ptr)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1137, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1355, in _do_run
    options, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1374, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

Caused by op 'MatMul_1', defined at:
  File "Iris.py", line 124, in <module>
    final_output = tf.matmul(first_output, A2)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 2064, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 2790, in _mat_mul
    name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3271, in create_op
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1650, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

如果我尝试将任何值放在轴上，它会停留在：

E:\#PROJEKTY\#PROGRAMOWANIE\AI-Project>python Iris.py
2018-04-27 00:53:15.388129: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
<tf.Variable 'Variable_2:0' shape=(1, 1) dtype=float32_ref>
Tensor("MatMul:0", shape=(?, 1), dtype=float32)
Traceback (most recent call last):
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 686, in _call_cpp_shape_fn_impl
    input_tensors_as_shapes, status)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid reduction dimension 2 for input with 2 dimensions. for 'norm/Sum' (op: 'Sum') with input shapes: [?,1], [1] and with computed input tensors: input[1] = <2>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Iris.py", line 120, in <module>
    euclid_dist = tf.norm(inputs_with_weights - c, axis = 2, ord='euclidean')
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\linalg_ops.py", line 552, in norm
    tensor * math_ops.conj(tensor), axis, keepdims=True))
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1373, in reduce_sum
    name=name))
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 5436, in _sum
    name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3273, in create_op
    compute_device=compute_device)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3313, in _create_op_helper
    set_shapes_for_outputs(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2501, in set_shapes_for_outputs
    return _set_shapes_for_outputs(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2474, in _set_shapes_for_outputs
    shapes = shape_func(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2404, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'norm/Sum' (op: 'Sum') with input shapes: [?,1], [1] and with computed input tensors: input[1] = <2>.

我不知道如何更正代码。有人可以帮助我吗？

编辑：高斯实现：

def gaussian_function(input_layer):
    initial = math.exp(-SC*math.pow(input_layer, 2))
    return initial


np_gaussian_function = np.vectorize(gaussian_function)


def d_gaussian_function(input_layer):
    initial = -2*SC*input_layer * math.exp(-SC * math.pow(input_layer, 2))
    return initial


np_d_gaussian_function = np.vectorize(d_gaussian_function)


def np_d_gaussian_function_32(input_layer):
    return np_d_gaussian_function(input_layer).astype(np.float32)


def tf_d_gaussian_function(input_layer, name=None):
    with ops.name_scope(name, "d_gaussian_function", [input_layer]) as name:
        y = tf.py_func(np_d_gaussian_function_32, [input_layer],[tf.float32], name=name, stateful=False)
    return y[0]


def py_func(func, inp, Tout, stateful=True, name=None, grad=None):
    rnd_name = 'PyFunGrad' + str(np.random.randint(0, 1E+8))

    tf.RegisterGradient(rnd_name)(grad)
    g = tf.get_default_graph()
    with g.gradient_override_map({"PyFunc": rnd_name}):
        return tf.py_func(func, inp, Tout, stateful=stateful, name=name)


def gaussian_function_grad(op, grad):
    input_variable = op.inputs[0]
    n_gr = tf_d_gaussian_function(input_variable)
    return grad * n_gr


def np_gaussian_function_32(input_layer):
    return np_gaussian_function(input_layer).astype(np.float32)


def tf_gaussian_function(input_layer, name=None):
    with ops.name_scope(name, "gaussian_function", [input_layer]) as name:
        y = py_func(np_gaussian_function_32, [input_layer], [tf.float32], name=name, grad=gaussian_function_grad)
    return y[0]
# end of defining activation function
4

1 回答 1

0

该错误表明第一个参数

final_output = tf.matmul(first_output, A2)

不是矩阵。所以first_output是不行的。它来自上一行：

first_output = tf_gaussian_function(euclid_dist)

所以我们需要审查该功能tf_gaussian_function()，但这不在发布的代码中。我检查了它是否是错字，应该是 tf.gaussian_function() 但没有这样的东西。

所以请发布函数的定义tf_gaussian_function()。

所以现在我们有了它，事实证明您正在尝试将函数math.exp( -SC * math.pow( input_layer, 2 ) )（通常称为 rho 或ρ）映射到euclid_dist; 我认为那将是您的径向基函数。旁注：看起来您正在尝试通过实现自己的渐变以及所有内容来完成一些繁重的工作np.vectorize()。我建议你实现一些更简单的东西，比如这段代码（未经测试）：

first_output = tf.exp( -SC * euclid_dist * euclid_dist )

（我使用x * x而不是等效的，tf.pow( x, 2 )因为第一个更有效。）这应该创建自己的渐变并照顾您想要实现的所有目标。但是无论如何，您听起来对您的实现工作很有信心，正如我在这里发现的那样；您还可以仔细检查它是否产生与此单线相同的结果。

就什么而言euclid_dist，tf.norm( x, ord = "euclidean", axis = None )返回第二范数（或欧几里得范数，也称为平方和的平方根），所以这绝对是一个标量。（它完全符合上面注释掉的行所说的：euclid_dist = tf.sqrt(tf.reduce_sum(tf.pow(tf.subtract(inputs_with_weights, c), 2)))。）不确定您要在那里实现什么？如果你只想要距离的平方，那么你应该这样做：

euclid_dist = ( inputs_with_weights - c ) * ( inputs_with_weights - c )

但据我了解，在径向基函数网络中，首先对输入应用权重，然后是径向基函数，第三步是欧几里得距离作为成本函数，这很可能是标量。

如果该行是您尝试为您的 RBF 网络实现规范化，请注意规范化通常会有所不同。因此，如果我正确理解了您的想法，您的实现可能看起来更类似于此：

from __future__ import print_function
import tensorflow as tf
from tensorflow.python.framework import ops
import numpy as np

x_data = tf.placeholder(shape=[1, 3], dtype=tf.float32)
y_target = tf.placeholder(shape=[1, 1], dtype=tf.float32)

# Create variables for NN layers
first_layer_nodes = 5
A1 = tf.Variable(tf.random_normal(shape=[3, first_layer_nodes]))  # input -> first layer nodes
A2 = tf.Variable(tf.random_normal(shape=[first_layer_nodes, 1]))  # first_layer nodes -> sum node
c = tf.Variable(tf.random_normal(shape=[first_layer_nodes]))  # centroids

# Declare NN
SC = 1
def rho( distances ): return tf.exp( -SC * distances * distances )
def norm( x ): return x / tf.reduce_sum( x, axis = -1 )
inputs_with_weights = tf.matmul( x_data, A1 )
print(inputs_with_weights)
distances = inputs_with_weights - c
print( distances )
first_output = norm( rho( distances ) ) # tf_gaussian_function(distances) # 
print(first_output)
final_output = tf.matmul(first_output, A2)

with tf.Session() as sess:
    sess.run( tf.global_variables_initializer() )
    r = sess.run( [ first_output, final_output ], feed_dict = {
                  x_data : np.array( [ [ 1.0, 2, 3 ] ] ) } )
    for v in r:
        print( v )

输出：

张量("MatMul_15:0", shape=(1, 5), dtype=float32)
张量("sub_8:0", shape=(1, 5), dtype=float32)
张量("div_5:0", shape= (1, 5), dtype=float32)
[[4.4366708e-03 6.8647589e-04 5.9621310e-01 7.5066246e-06 3.9865622e-01]]
[[0.31285414]]

要回复您发布此代码片段的评论：

exp_list = []
for i in range(first_layer_nodes):
    euclid_distance = tf.reduce_sum(tf.square(tf.subtract(x_data, c[i, :])), 1)
    exp_list.append(tf.exp(-SC * euclid_distance))
phi = tf.transpose(tf.stack(exp_list))

这可以被矢量化，利用隐式广播tf.subtract()和这样的高级索引（未经测试）：

ed = tf.reduce_sum( tf.square( tf.subtract( x_data[ None, ... ], c ) ), 2 )
phi = tf.transpose( tf.exp( -SC * ed ) )
于 2018-04-27T19:59:29.450 回答