Skip to content
Search

Caffe:Tour:Solver

lastmod 2018-08-17created 2015-11-183 edits

The solver orchestrates model optimization by coordinating the network’s forward inference and backward gradients to form parameter updates that attempt to improve the loss. The responsibilities of learning are divided between the Solver for overseeing the optimization and generating parameter updates and the Net for yielding loss and gradients.

Step

  1. Create the training network.
  2. Iteratively (iteratively) call forward and backward and update the parameters.
  3. (Periodically) evaluate the test networks (Test networks).
  4. Optimize and create snapshots.

In each iteration (iteration)...

  1. Call the forward (Forward) network to compute the output (Output) loss (Loss).
  2. Call the backward (Backward) network to compute the gradients (gradients).
  3. incorporates the gradients into parameter updates according to the solver method
  4. Go through the Solver Method to incorporate (incorporates) the gradients (gradients) into the improved parameters.
  5. Update the Solver state according to the LearningRate, History, and Method.

Methods

Solver Methods can generally solve the optimization problem of loss minimization (Loss minimization).

  • Stochastic Gradient Descent (type: "SGD"),
  • AdaDelta (type: "AdaDelta"),
  • Adaptive Gradient (type: "AdaGrad"),
  • Adam (type: "Adam"),
  • Nesterov’s Accelerated Gradient (type: "Nesterov") and
  • RMSprop (type: "RMSProp")

See also

Favorite site

sitemap