New Fast.ai Course: A Code-First Introduction to Natural Language Processing

jahbrewski|6 years ago

I just started the fast.ai MOOC (practical deep learning for coders), and boy, I much prefer reading through a tutorial than sitting through hours upon hours of videos. I looked for transcripts and/or a written version of the content and came up empty-handed.

Can anyone recommend a solid, written, introduction, covering similar material?

jointpdf|6 years ago

For deep learning / neural networks, I think these are the best balance of theory and practice (not to mention great visualizations): http://cs231n.github.io

More on the intuition/theory side for NNs, but Michael Nielsen’s book is highly recommended: http://neuralnetworksanddeeplearning.com

For NLP, you can try this extremely slick interactive course on spaCy (highly recommended Swiss Army knife library for NLP) from package author Ines Montani: https://course.spacy.io

The Keras tutorials are solid, here’s one on seq2seq models: https://blog.keras.io/a-ten-minute-introduction-to-sequence-...

For more cutting edge NLP, it looks like the fast.ai course covers the Transformer model (the basis of BERT, GPT-2, et al. This is a great overview of that architecture: http://nlp.seas.harvard.edu/2018/04/03/attention.html

And PyTorch implementations of Transformer models can be found here: https://github.com/huggingface/pytorch-pretrained-BERT/blob/...

LMK if anyone wants more, I have a learning resource hoarding problem.

EForEndeavour|6 years ago

Project idea: use the NLP skills taught in this course to build an automated system that converts the course's videos into a written tutorial.

sputknick|6 years ago

Maybe read their code and it's associated comments would help you? One of the benefits of using notebooks over a traditional IDE is that code is segmented into blocks. For me it makes it easier to read. it's like "ok, this cell will do this to our data, then this next cell will do this to our data". It's been more than a year since I took their class, and what I did to help with comprehension of the videos was to take notes in the individual cells while following the videos.

dragandj|6 years ago

(My own writings)

Deep Learning from Scratch to GPU at https://dragan.rocks

Deep Learning for Programmers (book in progress) at https://aiprobook.com/deep-learning-for-programmers

jsty|6 years ago

For at least some of the lessons there are pretty comprehensive notes / transcripts linked in the column on the right.

E.g. for https://course.fast.ai/videos/?lesson=2 there is https://github.com/hiromis/notes/blob/master/Lesson2.md

gumby|6 years ago

My gf worked for Coursera for a while so I tried some of their courses. The big win for me was that every one I tried included a full transcript. So my use model was to simply read the transcripts and if I didn't understand something I clicked on the text which would jump to that point in the video.

But in general for the stuff I want to learn I prefer to read a book, which is harder to find for really recent stuff like this.

thijsvandien|6 years ago

I have the same problem with video courses. The moment I start to sit and watch, I feel drowsy. It's strange, because I never had it so bad in real life lectures. Did anyone experience the same and find a way to fix themselves?

throwaway321546|6 years ago

https://www.deeplearningbook.org/ is also great

thellamafarm2|6 years ago

What is your skillset? What language?

codesushi42|6 years ago

Yes.

An Introduction to Statistical Learning https://github.com/tpn/pdfs/blob/master/An%20Introduction%20...

Deep Learning https://books.google.com/books/about/Deep_Learning.html?id=o...

Machine Learning Mastery books https://machinelearningmastery.com/products/

Convolutional Neural Networks from the Ground Up https://towardsdatascience.com/convolutional-neural-networks...

Transformers https://medium.com/inside-machine-learning/what-is-a-transfo...

Qiu_Zhanxuan|6 years ago