How to elaborate and train a Neural Network

March 31st, 2017 by JasonLe's Tech Leave a reply »

Elaborate a Neural Network

First, pick a network architecture; choose the layout of your neural network, including how many hidden units in each layer and how many layers in total you want to have.

  • Number of input units = dimension of features x(i)

  • Number of output units = number of classes

  • Number of hidden units per layer = usually more the better (must balance with cost of computation as it increases with more hidden units)

  • Defaults: 1 hidden layer. If you have more than 1 hidden layer, then it is recommended that you have the same number of units in every hidden layer.

构建 Neural Network 首先要明确要创建几个隐藏层,每个隐藏层有多少个参数。



Training a Neural Network

  • Randomly initialize the weights
  • Implement forward propagation to get hΘ(x(i)) for any x(i)
  • Implement the cost function
  • Implement backpropagation to compute partial derivatives
  • Use gradient checking to confirm that your backpropagation works. Then disable gradient checking.
  • Use gradient descent or a built-in optimization function to minimize the cost function with the weights in theta.

输入的特征的权重是随机指定的(如果全部输入的权重都为一个常数,那么输入到隐藏层的值就是相同的,那么导致hΘ(x(i))也是相同的,导致symmetry。不同的初始权重就是为了Symmetry Breaking)。