Elaborate a Neural Network
First, pick a network architecture; choose the layout of your neural network, including how many hidden units in each layer and how many layers in total you want to have.
-
Number of input units = dimension of features x(i)
-
Number of output units = number of classes
-
Number of hidden units per layer = usually more the better (must balance with cost of computation as it increases with more hidden units)
-
Defaults: 1 hidden layer. If you have more than 1 hidden layer, then it is recommended that you have the same number of units in every hidden layer.
构建 Neural Network 首先要明确要创建几个隐藏层，每个隐藏层有多少个参数。
首先输入单元个数就是输入的特征数，输出的个数就是分类的个数，每个隐藏层中单元的个数是多少？
通常意义上，隐藏层中单元的个数越多，这个分类效果越好，但是需要权衡计算与特征数的关系。默认情况，一个神经网络会存在一个隐藏层，当多余一个隐藏层的情况下，每层拥有的单元个数相同。
Training a Neural Network
- Randomly initialize the weights
- Implement forward propagation to get hΘ(x(i)) for any x(i)
- Implement the cost function
- Implement backpropagation to compute partial derivatives
- Use gradient checking to confirm that your backpropagation works. Then disable gradient checking.
- Use gradient descent or a built-in optimization function to minimize the cost function with the weights in theta.
输入的特征的权重是随机指定的（如果全部输入的权重都为一个常数，那么输入到隐藏层的值就是相同的，那么导致hΘ(x(i))也是相同的，导致symmetry。不同的初始权重就是为了Symmetry Breaking）。
实现前馈传播算法，计算出每层的x(i)，实现代价函数，通过反向传播算法计算每个Θ的偏导，然后通过梯度检查测试反向传播算法是否成功，然后将梯度检查disable掉（梯度检查计算复杂度太高）。
最后使用梯度下降找到最小的代价函数值和Θ。这个就是需要的特征集。
参考：
http://blog.csdn.net/jinlianjingd/article/details/50767743
https://www.coursera.org/learn/machine-learning/supplement/Uskwd/putting-it-together
