What makes the AWD-LSTM great?

The AWD-LSTM has been dominating the state-of-the-art language modeling. All the top research papers on word-level models incorporate AWD-LSTMs. And it has shown great results on character-level models as well (Source). In this blog post, I go through the research paper - Regularizing and Optimizing LSTM Language Models┬áthat introduced the AWD-LSTM and try to explain … Continue reading What makes the AWD-LSTM great?