inPace
All coursesكل الدورات

Deep Learning 5: sequence models, attention and transformers

  • about 10 h of lessonsحوالي 10 ساعة من الدروس
  • 9 modulesوحدات · 24 lessonsدروس
  • Englishالإنجليزية

Part 5 of a refresh of Andrew Ng's Deep Learning Specialization, with the original slides in each lesson and everything updated for PyTorch 2.15 and Keras 3.15: RNNs, language models and sampling, GRU and LSTM, word embeddings (word2vec, negative sampling, GloVe), sentiment and debiasing, sequence-to-sequence models, beam search, BLEU, attention, speech, and transformers, plus how they became today's large language models. Notebooks for Colab, as files and PDFs, and new charts from real runs. Plan: 9 days. Slides by DeepLearning.AI (CC BY-SA 2.0, non-commercial).

byمن إعداد inPace checkedموثّق

deep learningnlprnnlstmtransformersattentionpytorchkeras
Add to inPaceأضف إلى inPace Download fileحمّل الملف

On these roadmaps:ضمن هذه المسارات: Deep learning, with a machine learning refresh, Machine learning, from the maths up

What's insideمحتوى الدورة

Each module ends with tasks. The next module opens when they're done.تنتهي كل وحدة بمهام، وتُفتح الوحدة التالية عند إنجازها.

  1. Day 1: Recurrent neural networks 3 lessonsدروس · 65 min د
    • Start here10 min د
    • Sequence data and notationslidesشرائح20 min د
    • The recurrent networkslidesشرائح35 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  2. Day 2: Language models and sampling 2 lessonsدروس · 60 min د
    • Language modelsslidesشرائح30 min د
    • Sampling, and character-level modelsslidesشرائح30 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  3. Day 3: Vanishing gradients, GRU and LSTM 3 lessonsدروس · 70 min د
    • Vanishing gradients in RNNsslidesشرائح20 min د
    • GRU and LSTMslidesشرائح35 min د
    • Bidirectional and deep RNNsslidesشرائح15 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  4. Day 4: Word embeddings 2 lessonsدروس · 55 min د
    • Word representations and using embeddingsslidesشرائح30 min د
    • Analogies, cosine similarity and the embedding matrixslidesشرائح25 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  5. Day 5: Learning embeddings, sentiment and bias 3 lessonsدروس · 75 min د
    • Neural language models, word2vec and negative samplingslidesشرائح35 min د
    • GloVeslidesشرائح15 min د
    • Sentiment classification and debiasingslidesشرائح25 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  6. Day 6: Sequence to sequence, beam search and BLEU 3 lessonsدروس · 75 min د
    • Sequence-to-sequence modelsslidesشرائح25 min د
    • Beam search, refinements and error analysisslidesشرائح30 min د
    • BLEU scoreslidesشرائح20 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  7. Day 7: Attention, and audio 3 lessonsدروس · 65 min د
    • The attention modelslidesشرائح35 min د
    • Speech recognition and trigger wordsslidesشرائح20 min د
    • Looking back on the specializationslidesشرائح10 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  8. Day 8: Transformers 3 lessonsدروس · 90 min د
    • Self-attentionslidesشرائح30 min د
    • Multi-head attention and the transformerslidesشرائح30 min د
    • From the transformer to today's language models30 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة
  9. Day 9: Build it, and interview drill 2 lessonsدروس · 65 min د
    • The notebooks40 min د
    • Interview drill25 min د
    Practical taskمهمة عمليةExplain in your wordsاشرح بكلماتكQuizاختبار · 4 questionsأسئلة

Notebooks and downloadsالدفاتر والتنزيلات

They also come inside the course. Colab runs them free in your browser.وهي موجودة داخل الدورة أيضًا. يشغّلها Colab مجانًا في متصفحك.

Read a lesson firstاقرأ درسًا أولًا

Start here

Your plan

Days Module Slides
1 Recurrent neural networks C5 week 1
2 Language models and sampling C5 week 1
3 Vanishing gradients, GRU, LSTM, bidirectional and deep RNNs C5 week 1
4 Word embeddings C5 week 2
5 Learning embeddings, sentiment and bias C5 week 2
6 Sequence to sequence, beam search and BLEU C5 week 3
7 Attention, and audio C5 week 3
8 Transformers, and from transformers to today's language models C5 week 4
9 Build it, and interview drill notebooks

About 1 to 1.5 hours a day, for 9 days.

Code in this course is written for PyTorch 2.15 (October 2026), TensorFlow 2.21 and Keras 3.15. The slides use TensorFlow 1-style code in places; the notes show the current way.

How each lesson works: the slides for that part open at the top of the lesson. Read them, then the notes under them: the notes explain what the slide skips, show today's PyTorch and Keras code, and point to the experiments in the notebook. Notebooks are files that come with the course: open them in Google Colab (free, nothing to install), download them for Jupyter or VS Code, or read their PDF versions with all the outputs.

This is part 5 of 5: Sequence models, from RNNs to transformers, with notes on how today's large language models grew out of these ideas.

Credit: the slides are from the Deep Learning Specialization (course 5, Sequence Models) by DeepLearning.AI, taught by Andrew Ng, shared under Creative Commons BY-SA 2.0 for non-commercial educational use. Take the full course on Coursera for the videos and graded labs.

Maths is written in plain text: W·x is a matrix product, Σ a sum, ^ a power, [l] the layer, (i) the example.

Licenseالرخصة: CC-BY-SA-4.0 · Updatedآخر تحديث 2026-11-13

Slides: Deep Learning Specialization by DeepLearning.AI (Andrew Ng), shared by DeepLearning.AI under Creative Commons BY-SA 2.0 for non-commercial educational use. Notes, notebooks, charts, tasks and questions: inPace.

Add to inPaceأضف إلى inPace