What are batch, incremental, on-line, off-line, deterministic, stochastic, adaptive, instantaneous, pattern, epoch, constructive, and sequential learning?
• If you have to use standard backprop, you must set the learning rate by trial and error. Experiment with different learning rates. If the weights and errors change very slowly, try higher learning rates. If the weights fluctuate wildly and the error increases during training, try lower learning rates. If you follow all the instructions given above, you could start with a learning rate of .1 for batch training or .01 for incremental training. Momentum is not as critical as learning rate, but to be safe, set the momentum to zero. A larger momentum requires a smaller learning rate. For more details, see What learning rate should be used for backprop?” • Use a separate test set to estimate generalization error. If the test error is much higher than the training error, the network is probably overfitting. Read Part 3: Generalization of the FAQ and use one of the methods described there to improve generalization, such as early stopping, weight decay, or Bayesian learning. • Start with one
Related Questions
- What are batch, incremental, on-line, off-line, deterministic, stochastic, adaptive, instantaneous, pattern, epoch, constructive, and sequential learning?
- What are batch, incremental, on-line, off-line, deterministic, stochastic, adaptive, instantaneous, pattern, constructive, and sequential learning?
- Does the user have to sync off-line and on-line databases in MiMiC v3?
