Comments on: Understanding the Working of Universal Language Model Fine Tuning (ULMFiT)

By: NLP Metablog - A blog of blogs - ML-DL

NLP Metablog - A blog of blogs - ML-DL — Sun, 12 May 2019 06:29:50 +0000

[…] ULMFiT […]

LikeLike

By: Yashu Seth

Yashu Seth — Sat, 23 Feb 2019 14:24:11 +0000

In reply to kumar.

Yes, it can be used to classify any number of labels.

LikeLike

By: kumar

kumar — Tue, 19 Feb 2019 12:33:41 +0000

Thank you..is ULMFIT can be used to classify 4 labels

LikeLike

By: Yashu Seth

Yashu Seth — Sat, 16 Feb 2019 21:15:46 +0000

In reply to Ernest S Kirubakaran.

Let’s say we got h_1, h_2, h_3 ….. h_n hidden states from the last layer. These hidden states are combined as follows –
concat(max(h_1, h_2, …. h_n), mean(h_1, h_2, …. h_n), h_n)
Let’s call this vector h_final. h_final is now fed to a linear layer. The output of this linear layer (say, f1) is fed to another linear layer whose output dimension is 1 (in case of binary classification) and this gives the output (say, f2). f2 is a single real number. This f2 is fed to a sigmoid layer to get a number between 0 and 1.

LikeLike

By: Yashu Seth

Yashu Seth — Sat, 16 Feb 2019 21:03:30 +0000

In reply to kumarvrsec.

There are no pre-trained embeddings used. They are randomly initialized and learned during the language model pre-training step.

LikeLike

By: kumarvrsec

kumarvrsec — Fri, 15 Feb 2019 09:53:28 +0000

Can you explain what embeddings are used

LikeLike

By: Ernest S Kirubakaran

Ernest S Kirubakaran — Mon, 11 Feb 2019 13:10:42 +0000

Can you please explain more about concat pooling in the final classification layer? If we get multiple hidden states each from bptt length of input sentence, how are we going to get a final number between 0 and 1? Any paper explaining this approach would be great. Thanks in advance.

LikeLike

By: The year in AI/ML advances: 2018 roundup - AI+ NEWS

The year in AI/ML advances: 2018 roundup - AI+ NEWS — Sun, 23 Dec 2018 23:08:43 +0000

[…] by the idea of using language models, popularized this year by Fast.ai’s UMLFit (see also “Understanding UMLFit”). We have then seen other (and improved) approaches like Allen’s ELMO, Open AI’s […]

LikeLike

By: A Walkthrough of InferSent – Supervised Learning of Sentence Embeddings – Let the Machines Learn

Sun, 05 Aug 2018 20:45:59 +0000

[…] If you are interested in transfer learning, you should check out the recent work of Jeremy Howard and Sebastian Ruder on ULMFiT (here and here). […]

LikeLike

By: Links (18) | Nintil

Links (18) | Nintil — Sun, 08 Jul 2018 15:02:05 +0000

[…] Understanding the working of Universal Language Model Finetuning […]

LikeLike