Dense-Sparse-Dense Training for Deep Neural Networks

lastmod 2018-08-17created 2017-09-016 edits

DSD: Dense-Sparse-Dense Training for Deep Neural Networks

Training method

Nodes with small weights are removed from a Dense network trained with the conventional method, and the network is then trained again. This Sparse training process regularizes the model. Finally, restoring the removed nodes, initializing them to zero, and performing Dense training improves the model's performance without overfitting. DSD_Training_Diagram.png

Documentation

DSD - Regularizing Deep Neural Networks with Dense-Sparse-Dense Training Flow

DSD: Dense-Sparse-Dense Training for Deep Neural Networks

https://arxiv.org/abs/1607.04381

See also

• Deep learning

Favorite site

• DSD - lijiancheng0614
• sogangori - Dense-Sparse-Dense Training for Deep Neural Networks

Pre-Trained models

• DSD model zoo
• SqueezeNet-DSD-Training