A Disciplined Approach to Neural Network Hyper-Parameters: Learning Rate, Batch Size, Momentum, and Weight Decay – Paper Dissected

Training a machine learning algorithm requires carefully selecting hyper-parameters. But with neural networks, this can easily go out of control with so many things to tune. Besides, the optimal values of these parameters vary from one dataset to another.  Leslie N. Smith in his paper - A Disciplined Approach to Neural Network Hyper-Parameters: Part 1 - … Continue reading A Disciplined Approach to Neural Network Hyper-Parameters: Learning Rate, Batch Size, Momentum, and Weight Decay – Paper Dissected

Universal Language Model Fine Tuning ulmfit for text classification

Understanding the Working of Universal Language Model Fine Tuning (ULMFiT)

Transfer Learning in natural language processing is an area that had not been explored with great success. But, last month (May 2018), Jeremy Howard and Sebastian Ruder came up with the paper - Universal Language Model Fine-tuning for Text Classification¬†which explores the benefits of using a pre-trained model on text classification. It proposes ULMFiT,¬†a transfer … Continue reading Understanding the Working of Universal Language Model Fine Tuning (ULMFiT)