Deep Learning for Natural Language Processing

Deep Learning for Natural Language Processing

by Stephan Raaijmakers
Released November 2022
Publisher(s): Manning Publications
ISBN: 9781617295447

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Explore the most challenging issues of natural language processing, and learn how to solve them with cutting-edge deep learning!

Inside Deep Learning for Natural Language Processing you’ll find a wealth of NLP insights, including:

  • An overview of NLP and deep learning
  • One-hot text representations
  • Word embeddings
  • Models for textual similarity
  • Sequential NLP
  • Semantic role labeling
  • Deep memory-based NLP
  • Linguistic structure
  • Hyperparameters for deep NLP

Deep learning has advanced natural language processing to exciting new levels and powerful new applications! For the first time, computer systems can achieve "human" levels of summarizing, making connections, and other tasks that require comprehension and context. Deep Learning for Natural Language Processing reveals the groundbreaking techniques that make these innovations possible. Stephan Raaijmakers distills his extensive knowledge into useful best practices, real-world applications, and the inner workings of top NLP algorithms.

About the Technology
Deep learning has transformed the field of natural language processing. Neural networks recognize not just words and phrases, but also patterns. Models infer meaning from context, and determine emotional tone. Powerful deep learning-based NLP models open up a goldmine of potential uses.

About the Book
Deep Learning for Natural Language Processing teaches you how to create advanced NLP applications using Python and the Keras deep learning library. You’ll learn to use state-of the-art tools and techniques including BERT and XLNET, multitask learning, and deep memory-based NLP. Fascinating examples give you hands-on experience with a variety of real world NLP applications. Plus, the detailed code discussions show you exactly how to adapt each example to your own uses!

What's Inside
  • Improve question answering with sequential NLP
  • Boost performance with linguistic multitask learning
  • Accurately interpret linguistic structure
  • Master multiple word embedding techniques


About the Reader
For readers with intermediate Python skills and a general knowledge of NLP. No experience with deep learning is required.

About the Author
Stephan Raaijmakers is professor of Communicative AI at Leiden University and a senior scientist at The Netherlands Organization for Applied Scientific Research (TNO).

Quotes
I loved this author’s detail, his command of linguistics, and how he brought all the concepts together.
- Vamsi Sistla, Nike

Full of non-trivial, real-world examples that expose the real potential of NLP.
- Rohit Agarwal, Mobisy Technologies

Covers deep learning for NLP from the most basic concepts to the-state-of-the-art Transformers architecture.
- Giuliano Araujo Bertoti, FATEC

Perfect for novice and expert practitioners alike.
- Ritwik Dubey, Mastercard

Publisher resources

View/Submit Errata

Table of contents

  1. Deep Learning for Natural Language Processing
  2. Copyright
  3. brief contents
  4. contents
  5. front matter
    1. preface
    2. acknowledgments
    3. about this book
      1. Who should read this book
      2. How this book is organized: A road map
      3. About the code
      4. liveBook discussion forum
    4. about the author
    5. about the cover illustration
  6. Part 1. Introduction
  7. 1 Deep learning for NLP
    1. 1.1 A selection of machine learning methods for NLP
      1. 1.1.1 The perceptron
      2. 1.1.2 Support vector machines
      3. 1.1.3 Memory-based learning
    2. 1.2 Deep learning
    3. 1.3 Vector representations of language
      1. 1.3.1 Representational vectors
      2. 1.3.2 Operational vectors
    4. 1.4 Vector sanitization
      1. 1.4.1 The hashing trick
      2. 1.4.2 Vector normalization
    5. Summary
  8. 2 Deep learning and language: The basics
    1. 2.1 Basic architectures of deep learning
      1. 2.1.1 Deep multilayer perceptrons
      2. 2.1.2 Two basic operators: Spatial and temporal
    2. 2.2 Deep learning and NLP: A new paradigm
    3. Summary
  9. 3 Text embeddings
    1. 3.1 Embeddings
      1. 3.1.1 Embedding by direct computation: Representational embeddings
      2. 3.1.2 Learning to embed: Procedural embeddings
    2. 3.2 From words to vectors: Word2Vec
    3. 3.3 From documents to vectors: Doc2Vec
    4. Summary
  10. Part 2. Deep NLP
  11. 4 Textual similarity
    1. 4.1 The problem
    2. 4.2 The data
      1. 4.2.1 Authorship attribution and verification data
    3. 4.3 Data representation
      1. 4.3.1 Segmenting documents
      2. 4.3.2 Word-level information
      3. 4.3.3 Subword-level information
    4. 4.4 Models for measuring similarity
      1. 4.4.1 Authorship attribution
      2. 4.4.2 Verifying authorship
    5. Summary
  12. 5 Sequential NLP
    1. 5.1 Memory and language
      1. 5.1.1 The problem: Question Answering
    2. 5.2 Data and data processing
    3. 5.3 Question Answering with sequential models
      1. 5.3.1 RNNs for Question Answering
      2. 5.3.2 LSTMs for Question Answering
      3. 5.3.3 End-to-end memory networks for Question Answering
    4. Summary
  13. 6 Episodic memory for NLP
    1. 6.1 Memory networks for sequential NLP
    2. 6.2 Data and data processing
      1. 6.2.1 PP-attachment data
      2. 6.2.2 Dutch diminutive data
      3. 6.2.3 Spanish part-of-speech data
    3. 6.3 Strongly supervised memory networks: Experiments and results
      1. 6.3.1 PP-attachment
      2. 6.3.2 Dutch diminutives
      3. 6.3.3 Spanish part-of-speech tagging
    4. 6.4 Semi-supervised memory networks
      1. 6.4.1 Semi-supervised memory networks: Experiments and results
    5. Summary
  14. Part 3. Advanced topics
  15. 7 Attention
    1. 7.1 Neural attention
    2. 7.2 Data
    3. 7.3 Static attention: MLP
    4. 7.4 Temporal attention: LSTM
    5. 7.5 Experiments
      1. 7.5.1 MLP
      2. 7.5.2 LSTM
    6. Summary
  16. 8 Multitask learning
    1. 8.1 Introduction to multitask learning
    2. 8.2 Multitask learning
    3. 8.3 Multitask learning for consumer reviews: Yelp and Amazon
      1. 8.3.1 Data handling
      2. 8.3.2 Hard parameter sharing
      3. 8.3.3 Soft parameter sharing
      4. 8.3.4 Mixed parameter sharing
    4. 8.4 Multitask learning for Reuters topic classification
      1. 8.4.1 Data handling
      2. 8.4.2 Hard parameter sharing
      3. 8.4.3 Soft parameter sharing
      4. 8.4.4 Mixed parameter sharing
    5. 8.5 Multitask learning for part-of-speech tagging and named-entity recognition
      1. 8.5.1 Data handling
      2. 8.5.2 Hard parameter sharing
      3. 8.5.3 Soft parameter sharing
      4. 8.5.4 Mixed parameter sharing
    6. Summary
  17. 9 Transformers
    1. 9.1 BERT up close: Transformers
    2. 9.2 Transformer encoders
      1. 9.2.1 Positional encoding
    3. 9.3 Transformer decoders
    4. 9.4 BERT: Masked language modeling
      1. 9.4.1 Training BERT
      2. 9.4.2 Fine-tuning BERT
      3. 9.4.3 Beyond BERT
    5. Summary
  18. 10 Applications of Transformers: Hands-on with BERT
    1. 10.1 Introduction: Working with BERT in practice
    2. 10.2 A BERT layer
    3. 10.3 Training BERT on your data
    4. 10.4 Fine-tuning BERT
    5. 10.5 Inspecting BERT
      1. 10.5.1 Homonyms in BERT
    6. 10.6 Applying BERT
    7. Summary
  19. bibliography
  20. index

Product information

  • Title: Deep Learning for Natural Language Processing
  • Author(s): Stephan Raaijmakers
  • Release date: November 2022
  • Publisher(s): Manning Publications
  • ISBN: 9781617295447