4

在诸如 ImageNet Classification with Deep Convolutional Neural Networks 等论文中

http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf

训练方法似乎是带有随机梯度下降的基本反向传播。

尽管 CNN 是深度神经网络的一部分，但这仅仅是因为存在大量隐藏层吗？这是否意味着这里的反向传播属于深度学习的范畴，因为网络很深，即使它不遵循与使用贪心逐层训练（一种真正的深度学习技术）的 DBN 之类的模式相同的模式？

感谢您的帮助和建议。

4

3 回答 3

2

如果你阅读深度学习的维基百科页面，它说：“深度学习是机器学习的一个分支，它基于一组算法，这些算法试图通过使用具有复杂结构或其他结构的多个处理层对数据中的高级抽象进行建模。多重非线性变换”。

CNN 具有多层非线性变换，因此有资格作为深度学习模型。

同样在麻省理工学院http://www.deeplearningbook.org/的这本书中， CNN 也是深度学习模型的一部分。

DBN 和 CNN 之间存在一个重要区别，第一个是无监督模型，另一个不是，除了一个使用 DBN 进行预初始化。

如果您阅读了同样是深度学习模型的 RNN 或 LSTM，您会发现它们基本上都是使用一种称为随时间反向传播的反向传播的修改版本进行训练的。

因此，请记住数据中模型高级抽象的多重非线性变换的概念。

深度学习也指模型而不是训练。

于 2016-05-18T19:11:54.093 回答
0

according to my knowledge

1 neuron and 1 layer -> known as a perceptron

2 or more neurons, 1 layer, 1 input --> as an equation of a line y = w*x + w0

2 or more neurons, 1 layer, multiple input instances --> hyper plane

when you add the sigmoid functionality to the output of these neurons activations and combine these in another perceptron or multiple neurons you get a non-linear combination of hyper-planes.

the thing that make CNN to be called CNN instead of a simple Deep Neural Network is that you learn local neighborhood weights that can be in any part of the image. So there is a weight sharing process between neuron activations.

  • suppose you had N different 20x20 gray scale images and 5 hidden units in 1 layer,
  • if you were to implement a fully connected deep neural network, you would try learning a weight matrix of 400x5 - hence 2000 parameters - for the first hidden layer. and as the output you would have 5 dimensional vectors for each N image.
  • but in a cnn structure, you decide on a reasonable patch size in these 20x20 images, lets say 4x4, and learn 5 different 4x4 weights - hence 80 parameters. And as the output of this first hidden layer you would have 5 different 17x17 images for each N image. In other way of looking at the output of first layer is 1445 dimensional vectors for each N image. So it is like you learn less parameters but as the output you have more dimensions to learn from.

So when I look at your questions,

  1. If there was only 1 layer and the classification/regression was made after that 1 single layer it wouldn't be called DBN. But it could still be called CNN because there is the concept of 'convolution' and 'neural network'. And of course with 1 single layer there would be no 'backpropogation'.

  2. Backpropogation is needed when there is more than 1 layer present. Because the concept is to be able to backpropogate some error without having a 'real/expected output' which we can see in the middle layers. We only have ground truth for the last layer where we have our regression/classification. Hence, if I understood your question correctly, backprop here falls under deep network category.

  3. We can not say 'CNNs do not follow same pattern as DBN'. Because they certainly are DBNs with weight sharing functionalities.

I hope this answers your questions. And additionally I think it would be nice to point out the difference between DBN and DNN here.

Quoting from another website (https://www.quora.com/Deep-Learning/What-are-the-difference-and-relevance-between-DNN-and-DBN)

“Deep Belief Networks construct beliefs (probablilistical relationships between instances) based on unsupervised data, and then apply those relationships to a problem when presented with supervised data.

Basically, learn on unsupervised, where there is more data to learn from, then use on a problem.

Deep neural networks are simply large multilayer neural networks.”</p>

于 2015-01-08T00:03:58.577 回答
0

深度学习技术是特定卷积神经网络 (CNN) 中人工智能的最新技术，在模式识别、对象或人脸识别方面非常有效，许多库可用于 CNN，如Itorch、theano、Digits等，用于深入理解神经网络和深度学习点击这里

于 2016-03-17T06:41:56.163 回答