Tag: neural-network

A Disciplined Approach to Neural Network Hyper-Parameters: Learning Rate, Batch Size, Momentum, and Weight Decay – Paper Dissected

Yashu Seth2 Comments

Training a machine learning algorithm requires carefully selecting hyper-parameters. But with neural networks, this can easily go out of control with so many things to tune. Besides, the optimal values of these parameters vary from one dataset to another.  Leslie N. Smith in his paper - A Disciplined Approach to Neural Network Hyper-Parameters: Part 1 -… Continue reading A Disciplined Approach to Neural Network Hyper-Parameters: Learning Rate, Batch Size, Momentum, and Weight Decay – Paper Dissected

Understanding the Working of Universal Language Model Fine Tuning (ULMFiT)

Yashu Seth10 Comments
Universal Language Model Fine Tuning ulmfit for text classification

Transfer Learning in natural language processing is an area that had not been explored with great success. But, last month (May 2018), Jeremy Howard and Sebastian Ruder came up with the paper - Universal Language Model Fine-tuning for Text Classification which explores the benefits of using a pre-trained model on text classification. It proposes ULMFiT, a transfer… Continue reading Understanding the Working of Universal Language Model Fine Tuning (ULMFiT)